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ABSTRACT 


This 1S a comparison of two state-of-the-art large-scale nonlinear 
optimization systems exhibiting unprecedented problem solution capabilities 
both in size of problem handled and method of solution. These codes are 
MINOS, developed by B. A. Murtagh and M. A. Saunders, and XS, developed by 
G. G. Brown and G. W. Graves. The codes are evaluated with respect to 
their problem solving capabilities and potential for practical applica- 
tion by analysts. Computational results are presented for thirteen 
nonlinear and nonlinear mixed integer test problems with from two to 793 
variables (12 to 100 integer variables) and one to 401 constraints. 
Portions of this work were presented at the CORS/ORSA/TIMS joint meeting 
in Toronto, May 1981. 
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IT. INTRODUCTION 


This study 1s a comparison of two state-of-the-art nonlinear program- 
ming codes that are designed to accommodate problems of thousands of 
variables and constraints. While there are many codes designed to handle 
the general linear programming (LP) problem and its specializations, 
there are significantly fewer systems designed to reliably solve the much 
more difficult nonlinear programming (NLP) problem. Of these, very few 
are capable of solving "large-scale" problems: larger than, say, a 
thousand constraints or a thousand variables or more. Most of these 
large-scale codes are internal, proprietary systems developed by companies 
for the solution of their specific industrially related problems; the 
codes used by petroleum refiners for chemical process control provide 
Singular examples of such contributions. 

One of the codes evaluated in this study is MINOS (Modular In-core 
Nonlinear Optimization System) developed by B. A. Murtagh, the University 
Of New South Wales, and M. A. Saunders, Stanford University. The other 
is XS (X System) developed by G. G. Brown, Naval Postgraduate School, and 
G. W. Graves, University of California at Los Angeles. 

This is the first independent comparison of either code and is 
intended to serve both as an evaluation of each and as a guide to the 
potential user concerned with the applicability of each code to the 
individual problem with which he might be faced. 

Two caveats should be kept in mind while reading this evaluation. 


First, the codes are quite different in intended use. MINOS is intended 


1B 


as an academic production code and 1s designed to be readily distributed 
and applied by a wide variety of users. Extensive documentation and 
reliable performance have been paramount concerns in the development of 
MINOS. On the other hand, XS 1s used as an advanced experimental testbed 
for optimization research. The fully instrumented version used in this 
comparison is a prototype designed to be used almost exclusively by its 
Originators and their co-workers for a wide range of problems, such as 
large mixed integer and linear formulations and especially for decomposi- 
tion problems. As such, it 1S in a continual state of flux and varies 
considerably in its content (hopefully in an improving direction) from 
month to month. All results from XS are from the most recent prototype 
system at the time of publication cutoff for this thesis with no special- 
ization for nonlinear programming. Academic and industrial production 
versions of XS are typically customized to the application at hand and 
thoroughly documented for routine use. 

Second, although both systems are "large-scale" nonlinear codes which 
have been successfully used on many large, real-life problems, because of 
their intended day-to-day application, their characteristics are not the 
Same, nor are they intended to be. Therefore, any differences between 
them in speed or capability may be attributable to design intention rather 
than relative deficiencies in the algorithms, underlying data structures, 


Or implementation. 


A. GENERAL PROBLEM STATEMENT 


Tne general linear programming (LP) problem can be stated as: 
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a f 
minimize Cc X 


Sipyjeet to wt A 


where: 


Coe 


b, 6 


“< 
[A 
SI 


(objective function) 


(ranged constraints) 


xed (bounds on variables) 


variables; 

cost coefficients; 

constraint matrix coefficients; 
upper and lower constraint ranges; 


upper and lower variable bounds. 


The general non-separable nonlinear (NLP) problem can be stated as: 


minimize 
Subgeet to ae 
b 

where 
x 
f(x) 
g(x) 
oer 
b, 6 


B. COMPARISON CRITERIA 


g 
Ks 


(x) (objective function) 
x) <r (ranged constraints) 
a (bounds on variables) 


variables; 


general non-separable, nonlinear function; 


general non-separable, nonlinear constraint; 


i 


upper and lower constraint ranges; 


upper and lower variable bounds. 


In any study of this nature, one of the primary concerns is the 


criteria with which the codes are to be objectively compared. 


In this 


case, the guidelines recently published in Operations Research [Ref. 1] 


will be used with some modification to prevent comparisons that are not 


valid because of the somewhat different nature of the two codes. 


These 


criteria, to be elaborated in Chapter IV, are listed below. 
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1. Algorithm Capabilities 


This section contains a general overview of the types and classes 
of problems for which each code is designed and comments on the growth 
capabilities of each code. 

2. CPU (Compute) Time 

The CPU times listed are the virtual CPU times required for each 
problem running with precompiled load modules for each primary system 
code and do not include the linkage editor times. Since the problems 
have been run interactively on a virtual memory computer system, these 
times will vary somewhat from run to run depending upon computer loading. 
Extensive experience on the host computer with these problems indicates 
that the listed CPU times are valid within one per cent. Although the 
actual "clock," or “response,” time (as opposed to CPU time) varies as a 
function of system loading; empirical evidence gathered while conducting 
this study suggests that a useful rule-of-thumb is that actual clock time 
1S approximately four times as long as CPU time for the virtual memory 
time-sharing system used. This should provide a reasonable estimate of 
the response times to be expected for problems of this study. 

Because of the region requirements of the FORTRAN compiler used 
on the host computer (see Section I.C), one megabyte of default virtual 
memory was used for all problems in a single-step procedure. Comments 
concerning problem-dependent memory requirements of each system will be 
made in Chapter IV. | 

3. Number of Iterations 
The number of major iterations (linearizations) and pivots 


required to reach solution is given for each problem, with the caveat 
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that the nature of an “iteration” varies considerably between the algo- 
rithms. The specific nature of these iterations is discussed in 
Sections I1.A.1 and I1.B.1. 
4. Number of Function Evaluations 

The number of function evaluations to reach solution is listed 
for each algorithm. However, since this number includes both objective 
function and constraint evaluations, as well as gradient calculations in 
the case of MINOS, different amounts of information may be obtained on 
each function call and this may, therefore, be a deceptive comparison. 

5. User Friendliness 

One of the primary goals of this study is to evaluate the ability 
of a user familiar with some optimization theory but with little experi- 
ence with the individual codes to set up and successfully solve a problem. 
Because of the codes' different design motivations, it was expected from 
the beginning of the study that MINOS would be far superior in this 
regard. 

a. Ease of Setup 

One measure of the flexibility of a problem-solving system 

is the ease with which it can be adapted by the general user to the 
particular problem/data structure at hand. 

b. Debug Output 

During initial debugging of a problem, varying quantities 

and types of diagnostic information may be required to isolate a par- 
ticular error. The ability of each code to provide a tailored output in 


concise, readable form for the user will be evaluated. 
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c. Failure Mode 
Since the perfect optimization code has yet to be developed, 
One measure of a code's performance is its ability to fail “gracefully," 
leaving the user in a posture from which he can recover without all of 
his effort being wasted. The information given to the user when each 
code fails is examined to evaluate its usefulness in further problem 
exploration. 
d. Robustness 
As an aid to the inexperienced user, codes should be robust 
in their default parameters to minimize the amount of "tuning" that need 
be done on most problems. At the same time, the parameters must have 
sufficient scope and power to allow the experienced user to exploit the 
Structure of difficult problems where interaction by the analyst is 


required in order to achieve a solution. 


Crees! PROBLEMS 

This study has been handicapped, as have other similar efforts, by 
the lack of suitable test problems to test the full capabilities of the 
codes. In this case, the need has been for large (several hundred 
variables or more) real-life problems. The author has been unable to 
secure such problems for which publication release is available and for 
which the data is in a form that is readily usable by both codes. This 
1S a continuing and widely recognized problem which has prompted the 
development of a number of nonlinear artificial problem generators. 
However, although these generators can produce arbitrarily large 
problems, it has been the experience of the developers of the X System 
that the randomly generated problems produced by the generators are 
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not realistic tests for optimization codes because they possess none of 
the specialized structure that is routinely found in real-life problems 
and which, in fact, is the very thing capitalized upon by good codes to 
produce their excellent results on large problems. 

Therefore, for the purposes of this test, the empnasis has been 
placed on real-life, or at least well-known problems, as opposed to 
generated problems, at the sacrifice of size. Thirteen problems were 
selected for this study, of which at least nine have been previously 
published. The variables in the problems range from two to 793 and the 
constraints from one to 401. The problems contain a mix of linear, 
nonlinear, equality, and inequality constraints. Also included are two 
problems with (12 and 100) integer variables which have never before been 


formally solved as nonlinear integer and nonlinear mixed integer programs. 


D. COMPUTER SYSTEM 

All computing has been completed in the W. R. Church Computer Center 
of the Naval Postgraduate School, Monterey, California on its installed 
IBM 3033 computers using the VM/SP timesharing system. The load modules 
for both optimization systems and for each respective problem generation 
Subroutine were generated with the FORTRAN IV (H Extended) compiler using 
the OPTIMIZE (2) option [Ref. 2]. All problems have been solved inter- 
actively in real-time on the computer system using precompiled load 
modules of the respective optimization systems linked with code and 
parameter data for the individual problems. This research has promoted 
development of an extensive real-time library of service routines to aid 


in the preparation, execution, and interpretation of large optimization 
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problems. Although neither optimization system has a truly interactive 
solution algorithm, each can be used with interactive parameter settings 
and with real-time monitoring of solution progress, providing a fertile 


research environment. 
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Il. DSSGRIPTRLON OF SYSTEMS 


A. MINOS 

The MINOS version discussed in this study is MINOS/AUGMENTED (alias 
MINOS Version 4.0) which has been developed as an extension of an earlier 
MINOS vintage which solved problems with nonlinear objective functions, 
but with strictly linear constraints. MINOS/AUGMENTED (henceforth called 
MINOS) is a general-purpose nonlinear programming system designed to 
solve large-scale optimization problems exhibiting linear and nonlinear 
constraints, linear and nonlinear variables, and exploiting sparsity 
(relatively few non-null constraint-variable interactions) and exclusive 
linearity of some constraints and variables (respectively expressed in no 
nonlinear terms). Nonlinear functions in a problem should be continuous 
with continuous first derivatives, but need not be separable. Integer 
variables are not accommodated. The user specifies nonlinear objective 
functions with one FORTRAN subroutine, nonlinear constraint functions with 
a second FORTRAN subroutine, while the linear portions of the objective 
function and constraints, ranges, bounds, and initial starting point (if 
any) are specified in standard "MPS Format" [Ref. 3]. 

MINOS employs an augmented Lagrangian algorithm to solve problems 
with nonlinear constraints. This algorithm uses a sequence of sparse, 
linearly constrained subproblems which are solved using a reduced-gradient 
algorithm. 

MINOS is intended (as cautioned by its developers) as an extension 


of (not a replacement for) commercial mathematical programming systems. 
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Consequently, MINOS does not possess many of the algorithmic options 
(e.g., dual simplex) or data revision and file handling capabilities 
common to various commercial optimization systems. A complete descrip- 
tion of the code can be found in the support documentation [Ref. 4, 5, 6]. 
1. Algorithm 

A discussion of the MINOS algorithm is logically divided into 
two cases: 

a. Linear Constraints 

Where only linear constraints are present, MINOS 1s designed 


to solve problems of the form: 


minimize f(x) + ctx 
subject to Ax ; b 
b<x<b 


where f(x) is a continuous, continuously differentiable function 
with gradient: 
v f(x) = (af /ax 5) = g(x). 

In general, the constraint matrix A 1s assumed to be large and sparse. 

The foundation of MINOS is an efficient and reliable imple- 
mentation of the revised simplex method for LP [Ref. 7]. This combines 
sparse matrix technology [Ref. 8: pp. 213-226] with stable numerical 
methods for computing and modifying a triangular LU factorization of the 
basis matrix B. A sparse LU factorization of the basis matrix 1s computed 
using the “bump and spike" algorithm of Hellerman and Rarick [Ref. 9: 
pp. 67-76], which is updated in a stable manner by the method of Bartels 
and Golub [Ref. 10: pp. 266-268]. 
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In order to extend the simplex method for (LP) to (NLP), 
Superbasic variables are defined in addition to the usual basic (dependent) 
and nonbasic (independent) variables. Both basic and superbasic variables 
may possess non-extremal values (between their respective bounds). In 
the reduced-gradient technique employed [Ref. 11: pp. 97-131], at a 
given iteration there are NS (Number of Superbasic) superbasic variables. 
They are free to move in any desirable direction which improves the value 
of the objective function, while the basic variables are adjusted to 
maintain feasibility with respect to linear constraints. If no 
improvement can apparently be made with the current set of superbasics, 
one (or more) of the nonbasic variables is selected to become superbasic. 
This increases NS and the process is repeated. If, at any time, a 
basic or superbasic variable reaches one of its bounds, that variable 
becomes nonbasic and NS is reduced by 1. By the usual "pricing" of the 
nonbasic columns, Lagrange multipliers for the current active constraints 
are obtained, which then indicate which nonbasic variables (if any) 
Should be released from their bounds. If required, they are moved from 
the nonbasic to the superbasic set, rather than from nonbasic to basic as 
in the more conventional simplex method. 

A stable implementation of a quasi-Newton method is used to 
optimize over the superbasic variables. This method uses a triangular 
matrix of dimension NS to approximate the reduced Hessian (a suitably 
transformed sub-section of the matrix of second derivatives, 

(ae f/x; 9x5)). In large problems (i.e., NS grows as large as 100 
or 200), the data region required by the quasi-Newton method becomes 


excessive and MINOS automatically substitutes the Fletcher-Reeves 
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Conjugate Gradient Method [Ref. 12: pp. 149-154], which consumes relatively 
less memory. The rate of convergence of the algorithm drops significantly, 
but must be accepted due to storage limitations. 


The linear constraint problem may be re-formulated as: 


minimize F(Xy) - Sob; 
Subject to 
- XN Xn 2 
P: [Ay A, bi] yx, | =0,b< |x | <b 
p 0 
S S 
given vf (xy) = g(Xy) 
where: 
Xn = the nonlinear variables (those that are directly 
involved in the function f(Xy))s 
Xp = the linear variables (the remaining part of x); 
0 = the right-hand side (RHS) variable, which has upper 
and lower bounds of -1.0; 
S = the slack or logical variables (one for each row of A); 
Sobj = 1X) + Co (linear objective function value); 
A 2 LAy A, J linear coefficients, partitioned as x, and x, ; 
b = right hand side (RHS), composition or r and r, where: 
r ot Avxy at AX, per 
2) ie 5 IT Ayxy + AX, <r 
ror r 3; otherwise; 
m = the number of rows in A; 
n = the number of columns in A; 
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nn = the number of nonlinear variables, the number of terms 
iN Xy3 
NS = the number of superbasic variables. 


At any particular stage, the n + 1 +m columns of [Ab I] are 
implicitly ordered as shown in Figure 1. The nonlinear variables may end 
up anywhere in B,S, or N. 

b. Nonlinear Constraints 

When a problem contains nonlinear constraints, MINOS does not 
necessarily satisfy the nonlinear constraints until an optimal solution 
1s achieved. Therefore, the nonlinear constraint functions may need to be 
defined outside their ranges. 

The problem must be expressed for MINOS in the following 


standard form: 


minimize f(x) + chy + dly (nonlinear objective) cb) 
subject to f(x) + Avy = by (nonlinear constraints) (2) 
Aox + Azy = by (linear constraints) (3) 
b< | <b (bounds on variables) (4) 
, _ 1 
where: ORs ee (x) 
"1 (x)I; 


and the functions f(x) should be smooth and have computable gradients. 
The components of x are the nonlinear variables and must preceed the 
linear variables y in the problem. The constraints (2) are the nonlinear 
constraints and must appear in the problem before the linear constraints 


(3). The general constraints may contain any type of inequality, and 


Zo 





a | | | 
[A Bel) P= | B | S | N | 
| | | 

Basic Superbasic Nonbasic 


where: P is a column permutation transformation of P. 


Fig. 1. MINOS Tableau Arrangement 


ranges may be defined for the constraints. Upper and lower bounds (4) 
may (and should!) be specified for all variables. 

The solution process [Ref. 13] consists of a sequence of 
"Major iterations;" at the beginning of each the nonlinear constraints 


are linearized at a current point Xu approximated to first-order by: 


~ 


F(x, xX) = F(x,) + d(x )(x - x, ) , 
which can be written as: 


f = f, + Jy (x - xy) ; (5) 


k 
Here, J(x) is the Jacobian matrix whose 4th element is af! (x)/ax 5, 


The objective function is also modified, producing the following 


subproblem: 


minimize £2 (x) af cl + dly = Mi (f - f) 


~- 


+ (1/2) o(f - Ff)" (¢ - f) (quadratic objective (6) 


function) 
subject to f + ALY = by (linearized constraints) 7 
Aox + A2y = bo (linear constraints) (8) 
Ba 5 <b (bounds on variables). (Sy 
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The objective function (6) is called an augmented Lagrangian. The 
vector dy 1s an estimate of the Lagrange multipliers for the nonlinear 
constraints, and the term involving po 1s a modified quadratic penalty 
function. If desired, the Lagrangian term of the modified objective 
function may be set to zero by the user. The penalty parameter po may 
also be controlled by the user. The problem (6)-(9) is, of course, 
stated in precisely the form required by the linearly constrained MINOS 
algorithm. 

A flowchart of the MINOS NLP solution process is given 
in Figure 2. 

fi (xX) de) are the final solution and multiplier estimates 


from the k°" 


subproblem, convergence 1s assumed to have occurred 
if the following conditions are true: 
Xr 1s an optimal solution to the subproblem; 
Xp satisfies the nonlinear constraints within a specified 
tolerance; 
dy 1s not substantially different from Ae4qd 
Xee] 1s an optimal solution to its subproblem; 
a basis change did not occur during solution of subproblem k + 1; 
the reduced gradient did not increase significantly during 
solution of that subproblem. 
The salient point is that Ke is checked for feasibility 
and then the final point i 1s checked for optimality. Since 
normally very few basis changes occur on the final subproblem (ideally none), 


the solutions will be virtually identical and the tests for feasibility 


and optimality will have been applied to essentially the same point. 
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Fig. 2. MINOS Iteration Flowchart 
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2. Code Structure 
MINOS is intended for use primarily as a "stand-alone" system 
which solves a problem or a sequence of problems and then terminates; 
however, MINOS can also be called as a subprogram. Figure 3 shows the 
Subroutine structure of MINOS. 
a. Stand-Alone Structure 
Some problem description and code tuning are provided to 
MINOS by means of a SPECS file which contains a list of keywords and 
values to define run-time parameters. Most of the data for a problem 
is provided by means of the standard MPS Format file. If the problem 
contains a nonlinear objective function, it is specified (as can be its 
gradient) by means of a FORTRAN subroutine called CALCFG provided by the 
user. If the problem contains nonlinear constraints, they are provided 
by the user (along with their gradients) in the form of a FORTRAN sub- 
routine called CALCON. The input data is processed in the following 
order: 
the SPECS file; 
the MPS file; 
a basis file (allows an initial basis crash); 
data read by CALCON on its first entry; 


data read by CALCFG on its first entry; 


data read by CALCFG on its last entry; 
data read by CALCON on its last entry. 
The MAIN program provides a single array of storage for the 
code. The GO subroutine is a control routine which calls subroutine 
MINOS for each problem to be solved in the input stream. This routine 


a 





MAIN 


Acquire 
Core 





MATGEN MINOS REPWRT 


Write 
Report 


Generate 
SPECS 
and 
MPS 





SPECS SPEC2 MPSIN DRIVER 


Read 
SPECS, 


Solve 
Prope. 


Decode Read 
SPECS MPS, 


Save 
Basis 
Files 


from Allocate 
ISCRCH Storage 
ISCRCH 





Fig. 3. MINOS Subroutine Structure 
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can be modified for individual applications. Subroutine MINOS and its 
Sub-programs communicate with the input files supplied to the code by the 
user (MPS and SPECS) and generate BASIS and SOLUTION files. MINOS also 
contains output parameters which define the stopping condition for the 
problem, the dimensions of the problem, and the positions in the workspace 
array where various subarrays are located. 

b. Use as a Subroutine 

Because of the modular nature of the code, it is relatively 
easy to modify the system for individual applications. Most modifications 
to tailor the system would be to the control routine GO; run-time 
parameters and constraint information would still be input via the MPS 
and SPECS files. The dotted lines in Figure 3 show how a Matrix Generator 
and Report Writer could be incorporated into the system to be used in any 
manner convenient to the user. 
3. Documentation 

MINOS documentation consists of three publications [Ref. 4, 5, 6]. 
They provide such complete coverage of the system that only rarely has 
access to the MINOS FORTRAN source code been required. 

a. MINOS User's Guide 

This document is for the original MINOS system, which did 

not accommodate nonlinear constraints. It 1s quite thorough and provides 
some of the mathematical foundations upon which the code is built. It 
has one section devoted solely to problems with nonlinear objective 
functions. The input data required by the user is also thoroughly docu- 
mented with detailed coverage of the options available in the SPECS file 


as well as some description of the "MPS Format" used by the MPS file. The 
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various input/output options involving the basis and solution files are 
addressed in great detail. A complete breakdown of the iteration log 
Output is given to allow the user to trace solution trajectory, if 
required. There is a good deal of coverage of hardware-dependent matters, 
Since there are some subroutines that are computer manufacturer or 
machine dependent. Example problems are also given with complete imple- 
mentation advice. 

b. MINOS/AUGMENTED User's Manual 

This manual serves as an adjunct to (not a replacement 

for) the User's Guide. It is intended primarily to update the User's 
Guide for solving problems with nonlinear constraints, but contains 
additional information as well. Some mathematical overview is given for 
the nonlinear constraint case, as well as the necessary additional 
commands in the SPECS file and the additional subroutine (CALCON) required. 
There is some alteration of the syntax and keyword meanings and options 
from the (previously described) MINOS User's Guide, and the coverage is 
not complete; therefore, both documents must be used for nonlinearly 
constrained problems. MINOS can most readily be applied by synthesizing 
both documents to create a "driver template" (such as those used by the 
X System) for input files and function subroutines; this shows the user 
most available options, and references those that cannot readily fit 
in the "template" format. The MINOS/AUGMENTED Manual also contains 
sample problems with complete input and output listings which provide a 


useful guide to the new user. 
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c. MINOS Distribution Documentation 
This document serves as a cover letter for the distribution 
of the code via magnetic tape. It discusses in detail the machine- 
specific requirements for successful installation of MINOS on the user's 
computer system. MINOS is currently available with machine-dependent 
routines tailored for Burroughs, CDC, DEC, Honeywell, IBM, and Univac 
computers. The documentation also describes a procedure to run test 
problems on the distributed tape which allow a complete check of the 
code. Problems 6 and ll of this study are included as test problems on 
the MINOS tape. Some minor changes that have been made to the system 
Since the publication of the two previous manuals are also presented. 
4. Implementation 
The first step in problem solution is one of proper problem 
formulation. The need for understanding the problem, determining 
proper scaling, and establishing sensible ranges and bounds cannot be 
overemphasized and will be discussed more fully in Section III. Ignoring 
these steps in haste will invariably haunt the analyst in the long 
run. 
a. CALCFG 
Subroutine CALCFG provides MINOS with explicit calculations 
of the nonlinear objective function and its gradient. There is an 
indicator variable in the calling arguments that indicates the first, 
last, or some intermediate call to the subroutine, providing it an 
Opportunity to make intiial or final calculations if required. 
MINOS allows substitution of numerical difference approx- 


imation for the analytical gradient of the objective function. 
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b. CALCON 
Subroutine CALCON serves the same purpose for the nonlinear 
constraints as does CALCFG for the objective function. However, the 


option for differencing of the constraints in lieu of analytic gradients 


is not presently available. The gradient functions must be explicitly 


provided. 
c. SPECS File 


The SPECS file consists of 80-column, card-image records 
that provide MINOS with the parameters of each problem and allow the user 
to set virtually every control parameter within the code. Each of the 
card records contains (in free format) a sequence of items that includes a 
first "keyword," an optional second "keyword," and a number representing 
the value of the parameter identified by the keywords. 

d. MPS File 

Constraint matrix data and labels for constraints and vari- 
ables are provided to MINOS via an MPS file, which (as the name 
implies) uses standard "MPS Format" [Ref. 3]. In nonlinear problems, the 
only restrictions are that the nonlinear variables must occur in the 
left-most columns of the constraint matrix, and the non-linear constraints 
must be the top-most rows of the matrix. Additionally, an optional 
INITIAL bounds set may be specified which assigns initial values (if 
available) to the nonlinear variables in the problem. 

The need to state a problem formulation with proper scale for 


both functions and variables cannot be overemphasized. 
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e. Basis Files 
MINOS provides four different methods for loading or saving 
basis representations. These options are invoked by use of records in 
the SPECS file and allow the user to save current values/solutions and 
reload them at a later time using a number of alternatives. Because of 
the relatively small size of the problems in this study, these options 
were not extensively used, but appear to be well thought out and of great 
potential value in working with large real-life problems. 
f. Machine Dependent Routines 
MINOS was developed on IBM computer systems, and although 
every attempt was made to make the code as portable as possible using 
FORTRAN, some machine dependent details must be changed in order to 
install the code on other than an IBM machine. 
5. Output 
MINOS provides a great deal of flexibility in the detail of its 
Output through several SPECS file options as well as the opportunity for 
uSing a tailored report writer. 
a. Print Level 
The most direct control of output is via the PRINT LEVEL 
option of the SPECS file. This command, which can be thought of as a 
five-digit binary variable, allows the user to individually select 
any one or all of the following: 
invert statistics; 
the value of nonlinear variables; 


the Lagrange-multiplier estimates for the nonlinear constraints; 
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the values of the nonlinear constraint functions; 
the Jacobian matrix. 

In addition, the LOG FREQUENCY command controls how often 
information is printed from the iteration log. This informs the user at 
specified intervals of the status of the optimization effort. 

b. Solution File 

The solution file can be printed on any device (determined by 
a SPECS file entry) and is a static preformatted report. It is also 
designed to be subsequently read from disk by a self-contained program 
which extracts and saves the values required by the user. This allows 
the user to make general use of the solution without first tailoring the 
entire output file or modifying the basic code with a tailored report 
writer. 

6. Debugging 
In general, MINOS is an easy code to work with as long as the 
user can interpret the iteration log and ancillary printouts in the more 
complex cases. 
a. Gradient Checks 

The requirement for explicitly calculating gradients for 
the constraints, and preferably for the objective function as well, jis a 
nuisance at best; but it can prove to be helpful in the early debugging 
phase of a complex problem. MINOS checks the analytic gradient, comparing 
the function and gradient information provided by the user for consistency. 
This check normally uncovers errors of coding and/or calculus, or gives 
the user at least some hope that the provided functions and gradients are 


correct. 
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b. Error Statements 
Although the diagnostic statements provided by MINOS are 

explicit and normally lead directly to an error, a few are not documented 
in the support literature at this time and can lead to some confusion. 
Consulting the source code is of some help, since it is very well docu- 
mented for the technically qualified reader. However, not even the 
source code review has been able to pinpoint the cause of all errors and 
has forced the user to occasionally resort to more traditional FORTRAN 


error-checking techniques. 


Bes OY OTEM 

XS has been developed since 1974 [Ref. 14] as a general-purpose 
optimization system of advanced design which serves both as a prototype 
testbed for research and as the fundamental computational foundation 
of many application packages utilizing optimization. 

XS is designed to solve large-scale optimization problems, with 
Special emphasis on mixed integer models. Decomposition features are of 
premier importance to XS at an aggregate level of detail. However, the 
embedded linear programming module has received the most design effort-- 
it 1s the heart of XS and exhibits many singular features including: 

hyper-sparse data representation [e.g., Ref. 15]; 

complete, constructive degeneracy resolution [e.g., Ref. 16: 
pp. 1-34]; 

basis factorization [e.g., Ref. 17: pp. 91-110]; 

elastic range constraints [Ref. 14]. 

The nonlinear feature of the X System is designed for use with 
large-scale models exhibiting some functional nonlinearity (non-separable, 


a5 





nonlinear objective and constraints are admitted). However, XS is 
designed to efficiently solve models for which nonlinearity is relatively 
mild, hopefully involving a subset of model variables and constraints. 
Large refining models provide an ideal nonlinear paradigm in this context. 

Design criteria for XS require that all other features be supported 
Simultaneously with the nonlinear feature (e.g., see Problem 12 with 
generalized upper bound (GUB) factorization and mixed integer (MIP) 
variables in addition to nonlinear features). This inflexible rule 
derives both from the wide variety of production applications currently 
using XS, as well as from the research philosophy of the developers. 

The basic nonlinear feature of XS 1s designed to encourage reliable, 
error-free problem input and solution, minimizing the total response time 
including problem preparation, debug, tuning, and interpretation of 
output. Accordingly, gradient functions are not required, nor are they 
even suggested as an attractive option to any but the most skilled user. 
Long experience has shown that coordination of gradient functions with 
problem functions reliably inflicts more frustration than all other steps 
in solving nonlinear models! The X System is designed for robust perfor- 
mance with automatic numerical difference approximation for all functions. 

Another notable nonlinear feature of XS is the pervasive use of 
robust problem-independent algorithm controls, such as a dynamic "trust 
region” for approximation of the problem functions, a conservative ray 
search, and heuristic internal tolerances to stabilize performance in the 
presence of “irregularities” such as function discontinuities (common to 
many models, especially those that are new and those that employ spline 


approximations [see Section II1.B] from imprecise data observations). 
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Also, use of the elastic ranges invites elegant formal incorporation of 
internal estimates for approximate data, prioritizing of constraints, and 
various relaxation methods. 

XS completely consists of open FORTRAN subroutines. FORTRAN IV 
(H Extended) [Ref. 2] is the implementation dialect and an IBM 3033 the 
host computer. XS is installed and operating on other systems with 
compatible arithmetic architecture, and the FORTRAN subset actually used 
is widely accepted, even by FORTRAN 77 [Ref. 18]. In-core, out-of-core 
procedures, if invoked, require the most installation-specific preparation. 

Problem input may include any combination of explicit function 
definitions, sparse row and/or sparse column generators, and MPS Format. 
No tuning parameters, gradients or other problem-specific data are 
required other than some adequate (or default) estimate of sheer problem 
dimensions (rows and columns). An example of the minimum information 
necessary to invoke the mlaoaenn for a sample problem (Problem 9) is 
provided in Figure 4. 

Input may use any of several standard FORTRAN subroutine "templates" 
which require only rudimentary programming skill. Alternate interface 
procedures may exercise XS directly for extremely challenging problems. 
All relevant tuning defaults and tolerances are available for user 
Override via a simple menu in FORTRAN subroutine form. 

XS employs an (NLP) technique which generates a sequence of local 
linear programs (LP), each from previous solution and gradient estimates. 
Each local LP determines a search direction based on a first-order local 
approximation of the problem functions and on the dual variable bounds 


(elastic range penalties), forming a piecewise linear Lagrangian gradient 
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Formulation: 


maximize eo 
2 2 
subject to Yt + Y2 = ] 


Yy2 Yo 2.0 
Implementation: 
M Sia (number of constraints) 
N = 2 (number of variables) 


G(1) = Y¥(1)**2/900.000 + Y(2)**2/529.0D0 (constraint) 
GO = Y(1) * Y(2) (objective function) 


Fig. 4. X System Problem Generation in Explicit Mode 


and investigating this general search direction with a local, closed- 
interval ray search. 

XS reliably solves NLP's with excellent efficiency. More important, 
XS can be tuned for particular classes of problems to provide unparalleled 
solution speed. However, XS employs a first-order descent method, and 
one would expect that convergence to more than 3-4 decimal place objective 
function tolerance will occasionally require disproportionate effort. 
The designers have consciously opted for fast, easy, absolutely reliable 
convergence to 3-place precision, a tolerance frequently exceeding 


coefficient data resolution. Although much better solution resolution is 
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desirable, and routinely achieved for amenable problems, it is not 
guaranteed by XS without some experience (tuning). 

For those cases in which extreme solution tolerance is demanded, 
a second-order variation is available which augments the explicit con- 
straints with the first-order stationary conditions. This method has 
produced excellent results [Ref. 19], but the additional problem set-up 
time 1s seldom economically justified in lieu of simply running default 
first-order XS a bit longer. (This represents a classic paradox in 
evaluating algorithms and their actual application.) Given the additional 
set-up effort, robust hybrid first- and second-order descent algorithms 
are easily implemented by use of the constraint penalties as descent 
forcing functions. 


1. Algorithm 
The X System requires that (NLP) be restated elastically: 


(ENLP) minimize g?(y) cay) meee Za ch ZT 
subject to giy) +sta -rer 
byGy <b 


ania OU 
or, in an equivalent Lagrangian form: 
minimize NYT ae 9d 


Subgect to Dycey < D 


Ze Try eet = 0 
with Zee a g.(y) crear 


0 ; otherwise . 


eo 











We see immediately that z, z are interpreted both as constraint penalties 
and as linear dual bounds, and that convexity is preserved with z, Z 20; 


Given some solution for (ENLP), 


b<y' <6, 


a local elastic linear program is generated: 
) 
) °) 


, 
ay +z (vg. {y jay - 1) 


5 


(LELP) minimize vg°(y 


subject to bb < 


2 


ore 0 
with Zs oe vg. ly jay <r eae g, ly 


Saws 
zs if wgi(y Jay > =F, - g.(y®) 
aya 
QO 3; otherwise 

where Db, 6 are local bounds derived iis. b and enforced as a local 
"trust region" over which the functions g are approximated acceptably by 
first-order Taylor series and Z Z are local penalties derived from 
Z, Z (and optionally other artifacts of solution). 

The complete algorithm performs a ray search on the piecewise 
linear convex (for convex functions g) descent direction ay, updates 
y®, and generates another local elastic program if terminal conditions 
are not satisfied (see Figure 5). In particular, descent is assumed 
1: 

a). The latest improvement in the objective function is 


greater than a (specified) tolerance, and 


b). The maximum number of (specified) iterations has not been 


exceeded. 
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Note: Integer detail ommited from NLP solution process for 
clarity. 


Fig. 5. X System Iteration Flowchart 


41 








Convergence is assumed if there 1s no descent for a (specified) number 
of iterations. 

At each iteration, the local trust region is determined dynam- 
ically based upon (specified) minimum and maximum desired step-length (in 
y-units). It is increased in size if descent has been achieved, decreased 
otherwise. In addition, each local bound is modified by a (specified) 
factor if oscillatory behavior is observed. 

2. Code Structure 

The X System is primarily designed and used as an "open sub- 
routine” system, but can be run "stand-alone" for problems of specified 
structure. Figure 6 shows the module organization of XS. 

Default tuning parameters are defined in the NUCLEUS. For 
problems of given structure, a "template" is prepared consisting of 
two entry points. PROB initializes problem generators and specifies any 
non-default initial parameters; FGE provides function values as required 
by the problem generators, and can also dynamically change any parameter. 

Standard problem templates include: 

Explicit Mode (FGE provides explicit statements of the problem 
functions in terms of y); 

Sparse Mode (FGE provides first-order function information only 
once, and computes nonlinear terms on demand); 

MPS Mode (Sparse Mode with MPS input); 

Super-sparse Mode (PROB and FGE communicate directly with internal 


XS data structures). 
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Fig. 6. X System Module Structure 
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3. Documentation 

User documentation for XS 1s terse. A brief summary of the 
system and examples of LP, MIP, and NLP formulations, templates, set-ups, 
and solutions are given [Ref. 20]. The mathematical foundations are set 
forth in [Ref. 21]. Finally, the designers provide an internal mainte- 
nance specification and logic manual with full current implementation 
details and data structures. 

Complete documentation for XS is prepared for particular produc- 
tion applications [e.g., Ref. 22, 23]. Also, user-friendly interactive 
templates permit "classroom use" of XS in real-time with very little 
training. However, full documentation for prototype XS is not likely to 
appear until the system stops improving. 

4. Implementation 

Problem preparation for XS 1s straight-forward. First, a template 
should be selected which matches the requirements of the problem at 
hand. 

a. Mode Selection 

Explicit Mode -- Adequate for small and intermediate-sized 
problems, very easy to use, but a bit less efficient than other, more 
demanding modes. Efficiency 1s enhanced if nonlinear constraints and 
variables are identified for XS in a contiguous block. 

Sparse Mode -- Desirable for large problems and relatively 
easy to use. Efficiency is enhanced if non-null coordinates of nonlinear 
constraints and variables are identified. 

MPS Mode -- Required for problems from outside sources 


expressed in MPS Format. This mode also provides one means of introducing 
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labels for functions and variables using proprietary matrix generators, 
identifying non-null coordinates of nonlinear constraints and variables, 
and providing a PENALTIES section in an easy format. However, processing 
MPS Format files is relatively expensive, and editing requires another 
file with revision instructions. 

Super-sparse Mode -- Greatest efficiency and risk. Mandated 
only by huge problems, complex generators, or other uncommon model 
curiosity. 

Once the proper template is selected, the following are 
assembled, as required. 

D. reas 

Subroutine PROB defines initial conditions (connects MPS 
files as necessary) and sets up problem generators. Default tuning 
parameters may also be redefined. 

G. ‘“RGE 

Subroutine FGE generates function values for a given solution. 
The calling discipline provides a single argument which invokes complete 
function evaluations, only nonlinear function evaluation, only non-nul | 
coordinate function evaluation, or only non-null nonlinear function 
evaluation, depending upon the template adopted. Except for super-sparse 
mode, the solution, base (partially evaluated) function values and 
complete function values are all available in precisely the organization 
(order) specified by the user. In super-sparse mode, basis factorization 
and permutation vectors may be exercised directly by the user, as well 
(not suggested for the timid!). Tuning parameters may be redefined 


dynamically, as may any problem element. 
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d. MPS File 

This 18 required only in MPS mode. A PENALTIES section may 
be used, aS may an associated MPSEDIT file. Both files are in card image 
form with 80-character, fixed-length records. 

e. SCRATCH Files (problem independent options) 

These files are respectively required only for in-core, 
out-of-core use, for online storage of incumbent solutions in non-convex 
and decomposed problems, and for efficient reinversion of the basis. 
Unformatted FORTRAN Input/Output is used with the attendant file defini- 
tion requirements. 

f. CRASH File (problem independent options) 

This 1s required for basis crash options, for basis save 
options, and for efficient reinversion. Unformatted FORTRAN Input/ 
Output 1s used. 

g. Machine Dependent Routines (problem independent options) 

For IBM architecture, XS may use a high-resolution computation 
timer, an integer overflow arithmetic hardware interrupt trap, and a 
dynamic (GETMAIN) storage allocation. These routines are coded in 
assembly language. 

5. Output 
Standard output includes various (specified) levels of detail, 
ranging from absolutely nothing to complete logs of each primitive 
operation within each segment of each minor iteration. Frequently useful 
reports include: 
Input Summary -- by row and column giving labels (if any), basis 


factorization status, selection status, non-null element counts, maximum 
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magnitude and range of first-order elements, range and bound status and 
scale, and penalty scale. 

set-up Summary -- with tuning parameters, dimension keys, host 
environment, XS vintage, and intended problem template. 

Major Iteration Log -- with complete time and pivots, solution 
value (with and without penalty terms), ray search resolution, local 
gain, trust region size, and descent condition. 

Minor Iteration Log -- (optional frequency) with linear information 
as in Major Iteration Log. 

Exception Log -- with basis crash and stability recovery 
diagnostics. 

Solution Summary -- by row and column giving labels (if any) 
and complete elastic primal, or dual solution (or edited selected subsets). 

Final Summary -- with total pivots and pivots per iteration, 
total function calls and calls per iteration required by ray search, 
integer, and differencing operations, respectively, and total compute 
time. 

Solution File -- for automated real-time solution analysis 
[e.g., Ref. 24], or off-line report writing. 

In addition, a report writing template is provided for use in 
customizing output formats and report content. 

6. Debugging 

Nonlinear models fail most frequently in just three ways: 
programming errors in the problem representation fail to mate it compat- 
ibly with calling modules and data structures, or model errors for which 


a correctly coded problem representation generates an incorrect model, or 
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solution errors yield undesired stationary points from otherwise correct 
models. 

Some programming errors are almost completely undetectable by 
an open FORTRAN subroutine (e.g., subscript range errors, argument 
list omissions, etc.). Provision of standard templates is the best 
insurance measure for these perennial annoyances. Symptoms of such 
difficulties are usually severe, immediate, and fatal. A change of host 
language would ameliorate this situation, but benchmarks of FORTRAN 
competitors have not yet revealed a candidate with sufficient execution 
efficiency for economic use with XS. 

Since XS uses no analytic gradients (a profound advantage in 
model debugging), one need usually merely verify that a given solution 
y? produces function values correctly. The set-up and solution sum- 
maries of XS usually diagnose further trouble very quickly--the elastic 
range violations reported in any solution focus attention on likely 
candidate functions for further inspection. 

Of course, XS can minimize an objective function intended for 
maximization, and can produce stationary points for completely incorrect 
formulations and implementations; however, these are intellectual issues 
for the modeller to ponder more than bugs in the classic sense. No 
debugging aid can guarantee reliable solution of a model which is not 
understood by the user. 

Interpretation of terminal solutions sometimes reveals that 
a local stationary point has been selected. XS provides an alternate 
procedure for problems suspected of such behavior, whereby the objective 


function is included with the constraints, and its penalties used to 
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enforce successive level set descent at each major iteration (e.g., 
Problem 4 for which XS has found three stationary points). Non-convex 
problems usually also require that the ray search be tuned for extra 
effort. 

Some tuning advice is always a valuable part of solution debugging. 
For variables, XS uses a zero level and infinity value which may be 
altered. The trust region is used when unruly nonlinear function behavior 
(misleading first-order approximation) is predicted, or as an additional 
safeguard which costs little. A distortion factor is provided by which 
oscillatory behavior detected in successive solutions is damped. (This 
1s no substitute for conjugate directions or second-order information, 
but serves well in practice as an effective heuristic). 

A zero tolerance is used for functions, and may be altered. 
XS also requires an objective function tolerance for nonlinear convergence 
(and for integer convergence), as well as penalties for violation of each 
constraint range. Thus, the modeller must express a priori in terms of a 
Single objective function the consequences of violating feasibility and 
optimality. 

Default values for all tuning parameters are necessarily scale 
dependent and require reasonably well-scaled models for reliable perfor- 
mance. In this sense, the X system is rather unforgiving of users who do 


not express their models completely and carefully. 
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ITI. EXPERIMENTAL METHOD AND RESULTS 


A. METHOD 

In preparation for this study, a copy of the documentation 
[Ref. 4, 5, 6] and code for MINOS were received from Saunders (circa May 
1980, via G. Brown from a Vienna meeting with Saunders). The code was 
received on magnetic tape which contained the system source code (both 
Machine dependent and portable), test problem source code, assembler 
language source code, object module, and executable load module. The 
machine dependent routines, assembler code, and the object modules were 
all compatible for the IBM system on which this test was run. The 
OPTIMIZE (3) load module from the tape was not used, but replaced with 
one compiled at the Naval Postgraduate School with the OPTIMIZE (2) 
option [Ref. 2]. (OPTIMIZE (3) is not available at the Naval Postgraduate 
School at this writing.) 

The X-system was already in place on the Naval Postgraduate School 
computer and was used in situ. Its working "documentation" [Ref. 20] was 
also available. 

Because of the complete MINOS documentaion and the availability of 
several test problems already set up for immediate execution, al] problems 
were initially prepared for MINOS and then converted to XS (except as 
noted). This procedure required a greater initial outlay of effort (See 
Section I1.8) because of MINOS' requirement for gradient information not 
required by XS; however, the consistency checks make internally by MINOS 


of the function and gradient routines served as a troubleshooting tool to 
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ensure the functions had been correctly coded. As a result, the conver- 
sion of each problem from MINOS to XS took less than an hour in most 
cases using the computer system file editor, once the problems were 
successfully run with MINOS. The converse was not true, however, as the 
gradient functions required considerable time and effort to derive and 
implement for MINOS. 

Both systems are presently being developed, supported, and used 
by their authors on IBM 3033's. Therefore, neither system has been given 


an advantage by the computer used to support the evaluation. 


B. PROBLEM FORMULATION COMMENTS 

Although “easy" problems can be solved with brute force by either 
system, large or tricky problems require intelligent problem formulation 
for either system to be successful. Among the most vital guidelines to 
consider are the following. 

The analyst must UNDERSTAND THE NONLINEAR PROBLEM in order to success- 
fully solve it (or in order to assimilate the answer). This can be a 
major hurdle when solving someone else's nonlinear problem formulation or 
test problem if inadequate documentation exists to set up the problem in 
a format that is compatible with the optimization code being used. 
Although true for both optimization systems, in the case of XS this is 
absolutely vital. A good deal of the power of XS lies in the use of its 
"elastic" penalties on the constraints. If the problem is not fully 
understood, these penalties cannot be set intelligently and at best the 
power of XS is diminished, because the penalties require formal transfor- 
mation by the modeller of infeasibilities to units of the objective 
function. At worst, XS may not be able to solve the user-stated problem 
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at all since it views the elastic formulation as "the problem." This can 
be a major hurdle when solving someone else's nonlinear problem formulation 
or test problem if inadequate documentation exists to set up the problem 
in a format that 1s compatible with the optimization code being used. 
There is a tendency in many problem formulations to overlook points 
within the allowable variable bounds that generate singularities in the 
problem functions. By Murphy's Law! and O'Toole's Corollary,¢ it is a 
virtual certainty that any code will attempt to evaluate the functions at 
a Singularity if proper precautions are not taken by the analyst. 
These precautions may be in the form of restricted bounds on the variables 
that were not included in the original problem specification or in coding 
within the function generator. 
For example, in the region of a singularity a spline function may 
be substituted for the function of interest at some point such that: 
Endpoints are satisfied by the composite function; 
The composite function is continuous; 
The composite function has a continuous derivative; 
The composite function 1s a "reasonably" close approximation of 
the function and its gradient (e.g., a “majorizing function"). 
In the same vein, reasonable bounds for the variables should be 
specified to prevent excessive run times with either system. Even if the 
problem has been initially specified with unbounded variables, the intelli- 


gent analyst can restrict the problem to reasonable bounds. There are 


lig anything can go wrong, it will .. . at the worst possible 
moment . . . and in the way so as to maximize the damage. 


¢ Murphy was an optimist. 
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few real-life problems for which an infinite value for a variable 
makes sense. 

Both systems, in fact any useable nonlinear optimization code, are 
sensitive to scale and require that the analyst formulate the problem in 
a reasonable manner. In general, the data, variables, and functional 
values should be as close to 1.0 as possible. Due to efficient 
specializations of both algorithms for linear (or nearly linear) con- 
Straints, it 1s worth a good deal of effort by the analyst to minimize 
the nonlinearities in the problem by appropriate transformations. A 
nonlinear objective function is usually preferable to nonlinear constraints 
(if a choice exists via problem transformations). 

It is mandatory with XS and strongly recommended with MINOS that any 


initially specified solution lie within the bounds of the variables. 


C. PROBLEM DESCRIPTIONS 

The first eight of the test problems had been used for a previous 
study [Ref. 25] and were collected from a variety of sources. The first 
Six problems are listed in Himmelblau [Ref. 26: pp. 395-425] and the 
original source author/developer for each is listed. The next two are 
adaptations of an inventory model [Ref. 27] and an entropy model [Ref. 
28] which were specifically designed to illustrate real-world problems 
with few constraints but many independent variables, albeit with a 
relatively small number of variables compared with the capabilities of 
the two codes under test. The ninth problem was developed by the author 
during course work at the Naval Postgraduate School. 

Appendix A provides the mathematical problem statement, problem 
data, starting points, and an initial and final solution from each code 
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for all problems except Problem 11: the formulation of ETAMACRO provided 
as a test problem for MINOS by its developers did not exactly match the 
description of [Ref. 29]. Time considerations precluded resolution of 
the discrepancies, and although the MINOS test problem formulation was 
used, neither formulation is presented here. 

The remaining four problems were not solved by both codes either due to 
sheer time constraints, or in the case of Problems 12 and 13, because integer 
variables cannot be accommodated by MINOS. 

The alternate starting points for problems 1-8 were taken from Waterman 
[Ref. 25: pp. 56-91]. 

1. Problem 1 (Himmelblau 6) 

This problem [Ref. 30°] is an example of determining the chemical 
composition of a complex mixture under conditions of chemical equilibrium. 
It contains 45 independent variables and 16 linear equality constraints. 
MINOS returned a solution very quickly, although the solution from the 
second alternate starting point differed from the first two starting 
points. XS took a bit longer to reach a solution from all three 
Starting points; however, the objective value was improved over the MINOS 
solutions and the three solutions agreed with only minor variations. 

2. Problem 2 (Himmelblau 4A) 

This is also a chemical equilibrium problem which had been 
redefined in the Himmelblau study from a problem originally formulated 
and solved by Dantzig, Johnson, and White [Ref. 31: pp. 751-755] and 
discussed by Bracken and McCormick [Ref. 32: pp. 46-49]. It contains 10 


independent variables and 3 nonlinear equality constraints. Both codes 


SReference not viewed by author. 
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obtained essentially identical solutions from all three starting points 
in about the same amount of time. 

3. Problem 3 (Himmelblau 18) 

This problem was formulated by the Shell Development Company 

for the original Colville study [Ref. 33: p. 22] and consisted of 15 
independent variables and 5 nonlinear inequality constraints. MINOS 
returned a slightly better solution, though both codes returned solutions 
from all three starting points in approximately the same amount of 
time. 

4. Problem 4 (Himmelblau 16) 

This problem [Ref. 347] maximizes the area of a hexagon in 

which the maximum diameter 1s unity. There are nine independent vari- 
ables, 13 nonlinear inequality constraints, and a lower bound of zero on 
One variable. This problem contains many local stationary points and its 
solution difficulty is not reflected in the problem summary in Section D 
of this chapter. Both codes required a little tuning in order to avoid 
local minima. To verify both MINOS and XS solutions, an XS option was 
invoked which uses an alternate formulation of the problem using the 
Original objective function as a constraint, replacing it with an implicit 
objective function. (Recall that XS implicitly uses penalties in its 
objective function.) This constraint value then provides the actual 
objective function value in the final solution. Although the final 
results achieved for XS using the original formulation exceeded the 


alternate method in speed, alternate formulation appears to be very 


“Reference not viewed by author. 
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robust and seems to be a useful tool with much to recommend it for 
XS used on problems containing multiple local starting points. 
5. Problem 5 (Himmelblau 20) 

This problem [Ref. 35°] contains a linear objective function, 24 
independent variables, 12 nonlinear equality, two linear equality, and 
Six nonlinear equality constraints. The independent variables also are 
bounded to positive values. Both codes returned identical solutions from 
all three starting points, however, XS required more time to reach the 
solution. 

6. Problem 6 (Himmelblau 23--continuous relaxation) 

This is a weapon assignment problem formulated by Mylander [Ref. 
367] and presented by Bracken and McCormick [Ref. 32: pp. 22-26 | 
with 100 independent variables, a nonlinear objective function, 12 linear 
constraints and zero lower béunds for all variables. This problem was 
formulated for XS using the GUB option. Both codes returned approximately 
equal objective function values from all three starting points, however, 
the returned variable values were quite diverse, which comfirmed previously 
reported results [Ref. 25, 26]. XS required somewhat longer to reach the 
solution. 

Since it was suspected that the variability in reported results 
might be due to the integer nature of the variables in the original 
formulation which had been approximated with continuous variables by al] 
previous solution attempts, the problem was reformulated with integer 


(binary) variables using XS and solved again as Problem 12 below. 


Reference not viewed by author. 
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7. Problem 7 (Waterman 7) 

This problem was adapted by Waterman [Ref. 25] from an inventory 
model created by Choe and Schrady [Ref. 27: pp. 451-463]. The first 50 
variables represent the reorder quantity for 50 inventory items and the 
next 50 variables represent the reorder points for the same 50 items. 
The problem contains one linear and one nonlinear inequality constraint 
and 50 lower bounds on the variables. This was the most difficult 
problem solved by both codes. MINOS returned approximately the same 
solutions from all three starting points, however much more time was 
required from the alternate starting points. MINOS could not achieve a 
solution from the second starting point with the conjugate gradient 
option, but achieved much quicker solutions from the remaining two 
points. XS did not return as good a solution from any of the starting 
points and also required more time to solve the problem. 

8. Problem 8 (Waterman 8) 

This problem was adapted from an entropy model proposed by 
Scott [Ref. 28: pp. 204-211]. The nodes depict 46 population centers 
connected by a transportation network, represented by the connecting 
arcs. Using a congestion cost function, the model yields an equilibrium 
solution that identifies node populations as entropic functions of the 
total cost of transportation to a central place of work. The problem has 
46 independent variables, one nonlinear inequality and one linear equality 
constraint, and 46 lower bounds on the variables. MINOS returned the 
Same solutions from all three starting points. Solution times for both 


systems changed significantly with starting points. 
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9. Problem 9 (Dean 9--Game) 

This 1S an example of a small problem developed from a classroom 
exercise to serve as an example of how quickly problems of a very basic 
nature can be mae using the codes as time-saving tools. The problem 
has one nonlinear equality constraint and two independent variables. 

Both codes found the solution from al] three starting points in comparable 
amounts of time. It took less than an hour (clock time) from start of 
initial problem formulation until solutions were available from both 
codes. 

10. Problem 10 (Dean--Sortie) 

This problem is from current work being done by the U.S. Air 
Force based on a model done by Clasen, Graves, and Lu [Ref. 37]. It 
contains 81 linear constraints and 793 independent variables. Because of 
time constraints and an unsuitable data format for direct input to MINOS, 
the problem was only solved using XS. The GUB feature of XS was also 
used on this problem. The results are given to provide a measure of 
comparison of this relatively large problem with the other smaller test 
problems. Also, the run time for this problem on the host computer 
using the Air Force's present, but dated, solution algorithm is 321 
seconds. 

ll. Problem 11 (Dean--ETAMACRO) 

This is Manne's energy problem ETAMACRO [Ref. 29: pp. 1-45] and 
1s included on the MINOS distribution tape as a test problem. It contains 
401 linear constraints and 688 independent variables. Because the model 
was formulated in close conjunction with the developers of MINOS, the 


formulation is very compatible with that code's structure. However, it 
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was not suitably formulated to exploit the elastic features of XS and 
although the problem was solved in its linear form by XS, time constraints 
precluded a complete reformulation which would have been necessary in 
order to solve the problem efficiently on XS. It is presented here because 
of its relatively large size in comparison to the other test problems, in 
order to give some indication of MINOS' performance on large problems. 
12. Problem 12 (Himmelblau 23--Integer Nonlinear) 
This is a reformulation of Problem 6 with integer (binary) 
variables instead of the continuous approximations used in Problem 6. 
XS's GUB option was also exercised on this problem. Because of the 
integer nature of the variables, the problem was only run on XS. this 
problem has never before been solved as a nonlinear integer model. 
13. Problem 13 (Dean--Integer Nonlinear) 
This problem (Ref. 38: pp. 519-536] was submitted by Schmit 
to Graves and is an example of a current engineering design problem that 
1s highly nonlinear and contains integer variables. It has 48 nonlinear 
inequality and three linear inequality constraints and 12 independent 
variables, of which 8 are integer (binary). Because of the integer 
variables, the problem was solved only with XS, but is listed as an 


example of a nonlinear integer problem solution by XS. 


D. PROBLEM SUMMARY 

This problem summary lists results for MINOS using the default 
(on small problems) quasi-Newton method as well as the conjugate gradient 
option normally used only for large problems. This is done to present 
a complete comparison of the two codes using their respective large-scale 
algorithms. XS does not provide an automatic second-order descent method 
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option (although [Ref. 19] presents the groundwork for such an option 
without the "elastic" framework) since it would be of little use with the 


large problems for which XS is designed. 
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Primary Alternate Aiternate 

Solution So lution Solution 
Problem 1 (Himmelblau 6) 
MINOS 
(quasi-Newton) 
(bj. Fn. -1895.4 ~1895.9 -1906.5 
Linearizations II it l 
Pivots 49 50 49 
Fn. Evals. 16 he eS 
CPU time 0.07 0.06 0.06 
(Conj. Grad.) 
Obj. Fn. -1895.4 -1895.9 -1906.5 
Linearizations it l l 
Pivots 49 50 49 
Fn. Evals. 16 eS iS 
CPU time 0.09 0.0/7 Vie7 
XS (compact template) 
G54. Fn. -1910.0 -1910.0 -1910.0 
Linearizations 3 3 3 
Pivots 154 iO 145 
Allee Vals. 15i 150 150 
CPU time 0.39 0.36 0.41 





Primary Alternate Alternate 
Solution Solution Solution 
Problem 2 (Himmelblau 4A) 


MINOS 
(quasi-Newton) 


@bj. Fn. -47.761 -47 .761 -47 .761 
Linearizations 16 14 24 
Pivots 208 126 267 
pr. Evals. 1016 608 Lli@2 
CPU time el: 0.67 TS 
(Conj. Grad.) 

Obj. Fn. -47.761 -47.761 -47 .761 
Linearizations 16 14 24 
Pivots 501 281 496 
Fn. Evals. 3110 1620 3194 
CPU time 2.68 1.46 2./0 
XS (compact template) 

Obj. Fn. -47./66 -47 .762 -47.761 
Linearizations 4] 54 45 
Pivots 397 469 456 
Fn. Evals. 642 816 716 
CPU time 0.80 0.98 0.82 


Problem 3 (Himmelblau 18) 


MINOS 

(quasi-Newton) 

ji. Fn. -32 .349 -32.349 -32 .349 
Linearizations 7 8 9 
Pivots 76 61 93 
hia. ieVvalis. 329 264 403 
CPU time Owe2 0.29 0.43 
(Conj. Grad.) 

Obj. Fn. -32.349 -32.349 -32 .349 
Linearizations 8 12 9 
Pivots 2d 346 444 
Fn. Evals. 626 2110 2525 
CPU time 0.51 1.60 1.93 
XS (compact template) 

Ob). Fn. -34 .228 -38.651 -33 .546 
Linearizations 7 ) 18 
Pivots 100 102 in7 
Fn. Evals. 153 191 393 
CPU time Om O20 O57 
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Primary Alternate Alternate 

Solution Solution Solution 
Problem 4 (Himmelblau 16) 
MINOS 
(quasi -Newton } 
Gp). Fn. -0.86603 -0.86603 -0 .86603 
Linearizations / 9 20 
Pivots Bie 87 168 
Fn. Evals. 207 317 688 
CPU time 0.29 0.42 0.96 
(Conj. Grad.) 
Goa. Fn. -0.86603 -0.86603 -0.86603 
Linearizations / 9 20 
Pivots 78 120 324 
Pn. sEVals. 370 551 1698 
CPU time 0.39 0.55 1.6/7 
XS (compact template) 
(ej. En. -0.86606 -0.86605 -0 .86606 
Linearizations 28 32 29 
Pivots 356 340 356 
iieecvals. 413 Sn 427 
CPU time 0.40 0.51 0.41 
Problem 5 (Himmelblau 20) 
MINOS 
(quasi-Newton ) 
Obj. Fn. 0.05566 0.05566 0.05566 
Linearizations 6 Tal 7 
Pivots 52 104 62 
Fn. Evals. 48 129 85 
CPU time 0.63 ieZs 0.91 
(Conj. Grad.) 
ms). Fn. 0.05566 0.05566 0.05566 
Linearizations 6 slgt / 
Pivots 56 122 88 
Fn. Evals. 61 res 149 
CPU time 0.76 57 ie2 
XS (compact template) 
fag. Fn. 0.05566 0.05566 0.05566 
Linearizations 8 13 11 
Pivots 476 628 514 
Fn. Evals. 444 686 591 
CPU time 61 2.45 1.78 
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Primary 
Solution 


Alternate 
Solution 


Problem 6 (Himmelblau 23--continuous relaxation) 


MINOS 
(quasi-Newton) 


Obj. Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


(Conj. Grad.) 
bj. Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


XS (compact template) 


Ub). Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


Problem 7 (Waterman 7) 


MINOS 
(quasi-Newton) 


Obj. Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


(Conj. Grad.) 
Obj. Fn. 
Linearizations 
Pivots 

mn. EVvals. 

CPU time 


XS (compact template) 


by. Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


-1735.6 
1 


145 
314 
L.5 


-1735%6 
i 


243 
630 
2.6/7 


~1734.5 


38 
2424 
4068 
Sago 


80.744 
Ie, 

373 
2087 
28 .03 


80.744 
ne, 

662 
4657 
14.64 


Go. Con 
54 
4489 
SS 
20%! 


63 


30.0 


io 
343 
1296 


=) 7 35e0 
i 


Za 
742 
SweL 


-1735.0 


oi 
2194 
3303 
Saree 


e072) 
33 
1227 
6927 
FUOo 


No 


Solution 


80.726 
144 
14088 
15283 
67.68 


A\lternate 
Solution 


-1735.6 
1 


139 
343 
oS 


=1 7:35:40 
i 


Zo, 
742 
Smee 


-1734.6 


38 
2390 
3542 
Seon 


COn/ ay 
21 

784 
4231 
54.67 


80.727 
15 

507 
4041 
23 


90 .604 
117 
9207 
12798 
37 101 





Problem 8 (Waterman 8) 


MINOS 
quasi-Newton) 


Oey. Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


(Conj. Grad.) 
Obj. Fn. 
Linearizations 
Pivots 

Pm. EVals. 

CPU time 


XS (compact template) 


Ti FN. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


Problem 9 (Dean 9--Game) 


MINOS 
(quasi-Newton) 


bj. En. 
Linearizations 
Pivots 

Am. EVals. 

CPU time 


(Conj. Grad.) 
Oo). Fn. 
Linearizations 
Pivots 

Fn. Evals. 

CPU time 


XS (compact template) 


Up). Fn. 
Linearizations 
Pivots 

Pa. EValsS. 

CPU time 


Primary 


Solution 


-3.4685 


148 
718 
4.09 


-3.4685 


163 
991 
card 


-3.4519 
13 

787 
1225 
3.05 


345 .00 


2 
a2 


345 .00 


i 
a2 


344 .94 
33 

63 

293 
0.15 
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Alternate 
Solution 


-3.4685 
9 


231 
1163 
6.41 


-3 .4685 


240 
1417 
4.04 


-3 .4689 


1842 
3371 
8.02 


345 .00 


oF 


345 .00 


oY 


342.59 


44 


Alternate 


Solution 


-3.4685 
8 


146 
715 
4.06 


-3 .4685 


142 
884 
2.48 


-3.4643 
12 

676 
1174 
Cao! 


345 .00 


37 


345 .00 


Sy) 


345 .01 
10 
14 





Primary 
Solution 


Problem 10 (Dean 10--Sortie) 
XS (super-sparse) 


5). Fn. 2.00870+6 
Linearizations 5 
Pivots 977 
Fn. Evals. 2393 
CPU time 8.60 


Problem 11 (Dean 11--ETAMACRO) 


MINOS 

(quasi-Newton) 

my. Fn. 1338727 
Linearizations 1 
Pivots Tool 
Fn. Evals. 1926 
CPU time 41.36 
(Conj. Grad.) 

Obj. Fn. Bf 7 
Linearizations ih 
Pivots 32640 
Fn. Evals. 68868 
CPU time 1028.05 


Problem 12 (Himmelblau 23--Integer Nonlinear ) 


XS (compact template) 


Obj. Fn. -1734.2 
Linearizations 30 
Integer Linear. 3 
Pivots 285 
Fn. Evals. 211 
CPU time 4.41 


Problem 13 (Dean 13-- Integer Nonlinear) 


XS (compact template) 


Bij. Fn. 0.12000 
Linearizations 6 
Integer Linear. 3 
Pivots 63 
Fn. Evals. 145 
CPU time O55 
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IV. CONCLUSIONS 


A. ALGORITHM CAPABILITIES 
1. Type of Problems 

MINOS is capable of reliably solving both the general (LP) and 
(NLP) problems with any combination of linear and nonlinear constraints, 
but cannot accommodate integer variables. 

XS has the same capabilities as MINOS with the addition of the 
use of integer variables. XS can also employ decomposition and basis 
factorization. 

2. Growth Possibilities 

There 1S no inherent maximum problem size for either MINOS 
or XS. For sheer capacity, they are both limited by the computer memory 
requirements for their working arrays. However, as noted in Section 
II1.A.1 for MINOS, in large problems (when the number of superbasic 
variables exceeds 100 or 200) the shift to the conjugate gradient algo- 
rithm, which consumes less memory, results in a significant decrease in 


the theoretical rate of convegence of the algorithm [Ref. 5: p. 10]. 


B. CPU TIME 

MINOS was a bit quicker in solving many of the problems of this 
Study. While XS yields 2-3 decimal place precision in the objective 
function faster than MINOS, MINOS usually prevails in 3-5 place efficiency 


(even when using first-order conjugate gradient options). 
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C. STORAGE REQUIREMENTS 

For linear programs containing m general constraints, roughly 
100(m)-bytes of memory are required for workspace by MINOS. If there are 
many nonlinear variables, additional memory may be required. This 
workspace size may be adjusted by changing the size of one array in the 
main program for MINOS or by a non-FORTRAN routine that allocates storage 
at run-time. The choice of method is machine-dependent and guidelines 
are provided to the user in the documentation [Ref. 4]. 

XS, used strictly in-core, requires a region of approximately 
56(MN) + 8(NR) + 200K-bytes where MN is the total rows + cols, and NR is 
the size of the distinct real value pool. Storage requirements for 
nonlinear problems known to this writer are not a significant considera- 


tion for XS, or for MINOQS. 


0. NUMBER OF ITERATIONS 

Each major iteration of MINOS creates a local linearization of 
the nonlinear program, and then solves it after addition of a quadratic 
(augmented Lagrangian) objective function. XS simply solves local 
linearizations (with augmentation of the linear penalty function and 
local trust region). MINOS usually requires less of its iterations than 
does XS, but evidently works harder on each. 

Pivots (minor iterations) for MINOS represent classical basis exchanges 
(a superbasic variable becomes basic, replacing a basic variable which 
becomes nonbasic). XS pivots are counted in several varieties, including 
logical cases in which no basis change or objective function improvement 


takes place but the logical composition of the solution changes. XS 
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usually requires more of its pivots than does MINOS, but apparently 
executes each of them faster. 

Iteration counts do not appear to be adeuate general measures of 
efficiency for dissimilar optimization algorithms, or even alternate 


implementations of the same algorithm. 


E. NUMBER OF FUNCTION EVALUATIONS 

For the objective vunction precision specified in these tests, 
MINOS has generally required less function (/gradient) calls than XS. 
This difference can be crucial for problems in which function generation 
requires the vast majority of computing effort. However, a paradox is 
apparent in that such problems often have no closed-firm derivatives: 
desirable for (quasi-Newton) second-order algorithms .° In most cases, 
XS provides 2-3 decimal place precision in the objective function value 
with less function evaluations than MINOS, but MINOS produces 3-5 place 
precision with less function calls than XS. MINOS usually requires more 
function evaluations than XS to yield a feasible solution. 

For most classes of functions (polynomial, exponential, etc.), the 
gradient is of the same class as the function; the function calls (and 
therefore the required CPU time) to determine gradient information are 


approximately the same for both analytic and numerical methods. 


O4 conversation late in this research with Professor G. Vanderplatz, 
Dept. of Mechanical Engineering, Naval Postgraduage School, revealed 
that his widely used optimization codes for engineering design problems 
have been evaluated using the number of function calls as the exclusive 
gauge of efficiency, that closed-form gradients are rarely available for 
such problems, and that empirical engineering data resolution may often 
be as poor as + 20 percent. 


68 





fe USER FRIENDLINESS 

Because of its documentation and intended use, it was expected from 
the start of the study that MINOS would be the system most amenable to 
the user and the results bear this out. 

1. Ease of Setup 

This was one area in which MINOS was not necessarily better. 

The requirements for providing analytic gradients proved to be very time 
consuming, notwithstanding their aforementioned use as a trouble-shooting 
aid. In addition, the need to refer to a number of documents for construc- 
ting the SPECS and MPS files (until "templates" were constructed) was a 
handicap to the new user. Once the programming aids were in place, construc- 
tion of a new problem became quite straightforward with a minimum of 

outside reference required. 

Although relatively little was required of the user for initial 
setup on XS, becoming familiar with the techniques of using "pointer" 
variables required time; especially with the dearth of documentation. 

Interestingly, as the user becomes more experienced with both 
systems, MINOS becomes easier to use with the exception of the gradient 
functions, while in some ways XS becomes more difficult as the profound 
tuning capability of the system becomes apparent and places more demands 
on the talents of the user/analyst (i.e., appropriate problem formulation). 

é. Debug Output 

Both systems are capable of producing voluminous debug listings 

to assist the user, and the only limitation is the knowledge and patience 


of the user in their interpretation. 
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; 


3. Failure Mode 

Both codes give comprehensive output to indicate why they 
fail. As long as the user is sophisticated enough in system use to 
understand the diagnostics, he can usually intuit the cause. 

4. Robustness 

Default values exist for all parameters in the SPECS file for 
MINOS and experience has shown these values to be very robust with little 
tuning required other than specifying the problem-specific size parameters. 
Although the explicit statement of the objective function gradient is 
recommended for MINOS [Ref 6: p. 18], experimentation with the objective 
function differencing option has revealed no significant change in 
solution values or CPU time between the explicit gradient and the numerical 
differencing representation. 

Many of the XS tuning parameters are dependent on the scaling of 
the problem, and their robustness is in direct proportion with the user's 
ability to provide a well-scaled problem. This just requires reasonable 
care in the original problem formulation, but for complex test problems 


presented in completed form, reformulation and scaling can be vexing. 


G. SUMMARY 

After 15 months of intensive use of both codes, it is apparent 
that both systems have achieved what their designers intended. MINOS is 
a well-documented, easy-to-use code that reliably achieves excellent 
results on the general (NLP) problem while demanding only moderate skill 
of the user. Its default parameters are robust and require minimal 
tuning to achieve satisfactory results. Although its input files can be 
somewhat cumbersome to manage, they are straightforward and unlikely to 
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Cause a confidence crisis for the inexperienced user. Constraint gradi- 
ents, on the other hand, can be arduous to prepare and debug, even for 
simple test problems. MINOS does not have some of the more sophisticated 
file editing and solution options, but its developers candidly admit that 
this was not a design goal. MINOS is not capable of handling integer 
variables, but again, was not intended to do so. MINOS appears to be a 
significant improvement over other contemporary codes, and a most useful 
tool for the modeller/analyst. 

XS, is also exactly what it is claimed to be: an experimental 
testbed for state-of-the-art optimization research. It is very easy to 
use, but 1s not intended primarily for the casual, inexperienced user. 
Nor is XS specifically designed for nonlinear models. XS provides many 
flexible file editing, problem representation, solution, and report 
Options, but is designed for efficient custom applications to particular 
classes of models. The default nonlinear feature of XS, operating with 
one of the problem representation "templates," provides a quick-turnaround 
modelling enviornment. 

XS 1s designed to accommodate problems with no analytic gradients, 
poor data resolution, approximated functions, and all the attendant 
difficulties of real-life optimization at large-scale. Problems should 
be formulated for XS with realistic range intervals and meaningful 
penalties associated with constraints. Accordingly, XS can encounter 
difficulties with artificial test problems presented in strictest equality 
form with default (rather than modelled) penalties. 

The large-scale nonlinear integer capability of XS, combined with 


basis factorization (e.g., GUB), decomposition facilities, etc., make it 
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truly unique in the field of large-scale optimization. It is capable of 
solving problems that no other system in the world known to the author 
can attempt. By achieving this, XS has more than fulfilled the goals of 


its developers. 
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APPENDIX A 


TEST PROBLEMS AND RESULTS 


Problem 1 (Himmelblau 6) 


Source: [Ref. 30] 
No. of variables: 45 
No. of constraints: 16 linear equality constraints 


Objective function: 


/ ny 
© ace ) | > * ik 
Minimize: f(x) = Xi [5K + In a 
k=l] jel 
X ik 
a. 
Constraints: 
/ ny 
9; (x) = | EF KX ijk aD = Q 71 = leeeee6 
k=] j=l 
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bis and Ci, 8 for Problem 1 


jk J 


1 jk 
I moo 2958 1 i 1 0.0 6 3 0.0 
2 0.281941 2 1 = los i 5 Zc405 
3 g/ J9233 3 1 I 8 3 0.0 
4 47 .00022 + 1 -36 .60 g 3 59 o0 
) 47 .02972 i 2 -10.94 10 : -21.49 
6 0.08005 2 2 0.0 11 3 -32 .84 
i 0.08813 3 2 0.0 2 3 6.12 
8 0.04829 4 2 0.0 13 3 0.0 
3 ORO1L55 9 2 020 14 3 0.0 
10 OAOZ11275 6 2 0.0 LS 3 -1.9028 
IL 0.0022725 / 2 0.0 16 S -2 .8889 
v2 0.0 8 2 2.5966 Ly 3 =6..0022 
nS 0.0 9 2 =59'o0 18 3 -/7.4854 
14 0.0 10 2 Soleo 9 1 * = OG" 
Us 0.0 a GC -32 .84 2 + 0.0 
16 0.0 a 2 6.26 3 4 Zoi) 
tS 2 0.0 1 5 -16.79 
ll 3 10.45 2 3 0.0 
2 3 0.0 S 9 Sao 7/9 
3 3 -0.50 1 6 0.0 
4 3 OB 2 6 ieee 
5 3 0.0 1 / U0 
2 / aos 
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E.., Data for Problem 1 


yk 


1 
1 
1 
ae 
1 
1 
1 
1 
l 
1 
1 
| 
ee 
1 1 
1 it 
1 -l1 1 
1 
1 
1 
1 
i 
It 
1 
1 
1 
ee 
1 i 
1 ee! 
1 = il 
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BI 


NZ 


3 


14 


5 


16 





Xan ecm oemomemyeerG 9 %it0 di} i2@# 13 14 #215 ~ 16 
14,3 , 8 

15.3 1 ie =O ene | 

X16.3 2 1 -2 -2 

Bis an ae 

X18 3 4 1 -4 

X14 i it 

X04 i 

X24 -l 1 -4 

X16 1 it 

X06 1 

X26 -] 1 -4 
X16 1 

X06 1 -l 1 

X17 1 
X97 i -1 Hl 


Alternate Initial Points: 
a) X ax = 0.01 j=1,...,n, et. yf 


b) Xik = 
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Results for Problem 1 
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oer) eo oe SS) eo. oe 2 2 2 a2 2 & oO 020 0 2 0 020 Oo Oo 90 Oo OO OO © 


Initial 
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-1895 .4 
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oO 


oOo 2a 2 © Oo O&O Oo Oo OO WwW Oo © 
Ooo 2 6 © 


0 
0 
0 
0 
0 


SS oe >. 2 2 2 2 a2 2 Oo 0 6&6 OO OO OO © 


.64387 


nO SZ 


./26300-01 
a 
.88130D-01 
50 

707 9 

0 

0 

se 
.155000-01 


.80050D-01 
0 
.48290D-01 
5505) / 

ao) 

0 

.28194 
.211270-01 
0 


i, 


GW 
WwW 


— 
CJ 


SoS ©  &.©@ © 


SS.) 2] 2 2 2 @2 @2 © WwW © © 


0 
0 
at) 
0 
0 


eS 2.2, © 2&2 ©@2 Oo Oo 2&2 Oo 0 © 


m2 0-01 
.36270D-01 
a0) 

soo 

so 

a0) 

a0 
.155000-01 


.62771D-01 
.51860D-01 
.48290D-02 
.476 
.15170D-02 
0 

0 
.211270-01 
.11682D-03 








eS. 62. 2 0 0 BB OO OD G2 OC GG Go DD] a2 © 
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® e ® e ® e e e 6 e 
— FS FS FY RF RK RK PF RK RK FF ek Ke 
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.818 
-405 


ae 

aliZ 

gli 
m2 
085 
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.498 


NM Mm Ww NS Ww 


wo © © oO © oCoreaI a 2 0©O Oo Oo @G@ OO O@D Oo © 


> + 
SOON 


Oe 22 2 Oo =F © & Oo O 


Qo O82 © 


MINOS 


0 

7 

a0 
-22725D-02 
0 

0 
.90900D-02 
0 

C 

0 
.3636D-01 
a0) 

0 

0} 

oc I0 
.28194 
SZ 

.000 

030 
.80050D-01 
.88130D-01 
-48290D-01 
.15500D-01 
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.22/725D-02 
.@ 
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Seo oo eS 2S 2 Go © 2 2 Oo COC OO OD 


> - 
“SOON 


See ore Tose © Ga G2 © 


oS OS © 


XS 


. 10080D-02 
0 

nO 
.11477D-02 
a0 

= 
.55987D-02 
0 

30) 

i) 

0 
.22395D-01 
0 

20 

.65296 
.28194 

s1 O52 

.000 

.030 
.80050D-01 
.88130D-01 
-48290D-01 
.15500D-01 
.21127D-01 
-22725D-02 
Q) 
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Problem 2 (Himmelblau 4A) 


Source: [Ref. 31, 32] 
No. of independent variables: 10 
No. of constraints: 3 nonlinear equality constraints 


Objective function: 


10 10 
es 
Minimize: f(x) = > @ GC. + xX. - In 2 
i=1 i=1 
Constraints: 
x x x x x 
g, (x) = ¢@ 1, 2e Cy 2e on e J e 10 2=0 


Jo(x) =e +2e~* +e oS e' -1l1=0 
caoese sme +e —me +e -1=0 


where c) = -6.089 Co = -17 .164 C2 = -34 .054 Cy = -5.914 


-24.721 7 -14 .986 C4 = -24 .1000 


i 
ib 


-10.708 -26 .662 Cig = -22.179 


i 





Results for 


g, (x) 
g5(x) 
93(x) 


Alternate Initial Points: 


Problem 2 


Initial 


-21.015 


Ww WwW WW 


~ 


OW 


OW 


.298 
.499 
ES) 


cou 


-5.0 


MINOS 


-47 .761 
-3.2023 
-1.9124 
-0 24443 
-6.5612 
0.72310 
-7.2742 
—omoouo 
-4 0203 
-3.2884 
-2 3344 


Vs lems 5 LU) 
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XS 


quite 

.3543 
.9456 
.22381 
Io 

n/ CUD0 
alo 

Say (es, 
HIZ13 
.0806 
~4085 

. 10383D-01 
.21430D-02 
.151530-01 





Problem 3 (Himmelblau 18) 


Source: [Ref. 33] 

No. of variables: 15 

No. of constraints: 5 nonlinear inequality constraints 
15 bounds on independent variables 


Objective function: 


10 


5 5 5 
Maximize: f(x) = ) eX. = ) | > yz - 2 7. 42° 
ee 


1=1 j=l i=] 
where y = Ci 5%(10+i) 


and z= X(10+4) 


Constraints: 


Z 10 
2 ; yoo 3d.2¢ +e. - ) aeenx.a1> 0 je= 1) 
S ‘ 1 ° 

i=] i=] 
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Data for Problem 3 


-15 
30 


-10 


oe 


-10 


-16 


3 


Alternate Initial Points: 


Ni 


a0 
0 


i=l,. 


T= lee. 


-2/ 
-20 
oo 


-31 
oc 


sane 


82 


-36 
-10 


10 
-6 
10 
10 


-18 
oe 
-31 
-6 
oo 
-20 


-12 
-10 
32 
-10 
-20 
30 
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Results for Problem 3 


Initial 
f(x) 2400.1 
x 1.0D-04 
Xo 1.0D-04 
X 1.0D-04 
X4 1.0D-04 
Xp 1.0D-04 
X6 1.0D-04 
X 6.0D-01 
X9 1 .0D-04 
Xg 1.0D-04 
X10 1.0D-04 
xy 1.0D-04 
X19 1.0D-04 
X13 1.0D-04 
X14 1.0D-04 
X16 1.0D-04 
g, (x) 4.5D-01 
g5(x) 3.3D-01 
9 (x) 2.4D-01 
g(x) 4.2D-01 
Jo (x) 4.8D-01 


MINOS 


-32 .349 
0.0 


5.1740 


3.0611 
11.840 


O05 90 


. 30000 
. 33347 
. 40000 
.42831 
. 22396 


Son SS 7S OF eoe © Or Oo © 
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-34 


— 
NO GW 


So ona & a a Goh hU6cOlUmUlUClCETCCUCCOCLCLCUO 


Oo © 


XS 


necS 


ao OL 
algo7/.0 
.6463 

.886 

0) 

0 

0 

a0 
.20177D-01 
0 

a) 

. 36088 

. 26740 
.51426 
.14235 

50 

.22845 
.40583D-01 





Problem 4 (Himmelblau 16) 


Source: [Ref. 34] 

No. of variables: 9 

No. of constraints: 13 nonlinear inequality constraints 
1 upper bound 


Objective function: 


maximize: 
f(x) = 0.5(x4x, - XoXq + X3Xq - XpXq + XoXy - X 6X7) 
Constraints: 
1 - x5 > xf ZU 
ae co 0 
1 - Ke - Ke > 0 
1 - < Se oe pao 
1 - (X4 - xe) = (xX, - i> 0 
ee ne = Xan ce 0 
1 - (X. ~ eh - (xX, - oe > 
L = (x3 = xz)* = (xq - xg)* > 0 
1 - me = Xe oN > 0 
X4Xq 7 XoX3 2 0 
X3Xq > 0 
“XpXq > 0 
XpXq - XpXo pag 
Xq > 0 
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Results for Problem 4 


Initial MINOS 

f(x) 0.0 -0.86603 

X lee -0.78028D-01 
Xo 1.0 -0.67479 

X3 1.0 0.82437 

Xq ee -0 56605 

Xe 150 -0.78028D-01 
Xe 1.0 -0.99695 

X7 ao 0.82437 

Xo 1.0 -0.24388 

Xg 1.0 0.32216 

93 (x) -1.0 0.0 

9o(x) 0.0 0.89621 

93 (x) -1.0 0.0 

g4(x) 0.0 0.0 

Je (x) 1.0 0.89621 

96 (x) 1.0 0.0 

g7(x) ao 0.0 

Jo(x) 1.0 0.89621 

Ig (x) 0.0 0.0 

919(*) 0.0 0.60044 

944 (*) 1.0 0.26558 

91 9(x) -1.0 0.25138D-01 
943(x) 0.0 0.84089 


1.0 1 


fy 
—— 

x 

ii 


ess 


om 
Se 
~x< 

i 


520 la leiene een 
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oOo 2 &] © @esOa © 


OO) Or @ ©: i 


Or © © @ © 


XS 


.86581 

ooo l/ 41 

. 36276 
nLOcol 

. 98669 
puooy 3 

so 0193 
.15144 
.98818 

nt) 
.73055D-04 
.0000 
.55734D-03 
.32201D-03 
732937 
.31116D-04 
.31858D-03 
.99987 
.5/312D-03 
.86025 

1G, 

a0 

Ro lo7 





Problem 5 (Himmelblau 20) 


Source: [Ref. 35] 
No. of variables: 24 
No. of constraints: 12 nonlinear equality constraints 
2 linear equality constraints 
6 nonlinear inequality constraints 
24 bounds on independent variables 


Objective function: 





24 
minimize: f(x) = ) aX, 
j=l 
Constraints: 
X(4+12) ea 
oe =>; - 7 HOM ean linet: 512 
‘i 4 
D( 4412) a j me », j 
j=13 j=l 
24 
Q13(x) a 
i=l 





——" ——— 
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Data for Problem 5 
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H © 
> MN 
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ah. 6 
56.708 
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80.8 
64.517 
49.4 
49.1 


Or 
UBS 
0.4 
Os 
0.6 
O. 





Results for Problem 5 


Initial 


Sono. ae o> 2 => 2 @2 ©. 2 2 © 2 2 2 © @ Oo 2 OO OO OO 


. 14696 
.04 
04 
04 
.04 
04 
.04 
.04 
.04 
.04 
04 
.04 
.04 
04 
.04 
04 
.04 
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.04 
.04 
04 
04 
04 
04 
.04 
.9D-01 
.2D-02 


So 2.2) 0 a2 ag 2 ©2 © © 


$ 
© 


MINOS 


.55658D-01 
a) 

ua 
39 
-40506D-14 
.11591D-13 
.22064D-14 
./5541D-01 
0 

0 
-174190-13 
0. 
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0.11195D-01 
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oOo 2] 2 Oo 0 0 2 2 02 O 


.10753D-16 
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Oo © © 
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a0 
~45547D-15 
-40622D-03 
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0 
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OF: 
Q. 
Or. 
i: 
Q. 
OF 
0, 
Q. 
.15094D-09 
.80735D-08 
.99860D-09 
.17049D-07 
.11195D-01 
.55220D-07 
SAIS 

.28861 

.62272D-09 
.39282D-07 
.23459D-10 
S286 

.14615D-09 
.29088D-08 
.14346D-08 
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-69666D-07 
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27513D-08 
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11439 

24201D-09 
28816D-08 
13956D-10 
7$9541D-01 





93 (x) -1.5D-02 
gy (x) 2.8D-02 
go (x) -5.6D-02 
96 (x) I ipea2 
94 (x) -2.4D-02 
99 (x) 5.9D-02 
9g (x) 4.3D-02 
91x) 4.10-02 
91 4(x) 1.3D-01 
94 9(x) 1.0D-02 
913(x) -4 .0D-02 
974(*) -7.3D-01 
915(X) 1.6D-02 
916 (x) 2.2D0-01 
917(x) 3.2D-01 
919(x) 2.20-01 
919(x) 5.2D-01 
9oq(X) peen-0l 
Alternative Initial Points: 
a) x. = 0.08 
b) <eOnOZ aaa 
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Solrce: [Ref. 32, 36] 
No. of independent variables: 


No. of constraints: 


Objective function: 
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Problem 6 and Problem 12 (Himmelblau 23) 


12 linear constraints 


100 lower bounds on the variables 


20 5 
,.. \ | “ty 
minimize: f(x) = u as 3 -1 
j= 7=1 
= Constraints: 
5 
) ~~ S = 
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20 
7 7 | Xj 7c 1 oe eee 
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Xig ve a ee jee eee 20 
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Data from Problem 6 
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Results for Problem 6 


Pine id! MINOS XS XS Integer 

x(") 100.0 as (see next page) — 

mex ) -1755.0 =79 5). 0 -1734.5 -1734.5 
9, (x) -1770.0 BOTS 30-236 50.0 
9o(x) -1800 .0 100 .00 100.0 100 .0 
93(x) -1660.0 Slews2 Bias 51.0 
gq (x) -1800.0 58 .824 Bees gc .0 
92(x) -1680.0 70.000 7030 7020 
I(x) 505.0 i261 48 .139 S20 
94(x) 410.0 62.414 60.401 61.0 
9g(x) 260 .0 -200.00 . -200.0 -200.0 
9g (x) S500 -100 .00 -100.0 -100.0 
949 (x) 130.0 -300 .00 -300.0 -300.0 
941 (%) 6151.0 -150.00 -150.0 -150.0 
919(x) 240.0 -250.00 -250.0 -250.0 


Alternative Initial Points: 


a) x13 * 10.0 Xo = 310 X34 7 eo. U 
Xqz = Loo X55 > iZs0 ieee. 20 
b) Xi > 10 tae Geers, lo oo ee 0 


Os 





Target 
j l 2 
l 
2 -13.5- -1.4- 
(353) (a5) 
40.0 iO 
3 
4 -23.5- 
(R77 ) 
20.0 
5 -20.9- 
Slens) 
L250 
6 -100.0- 
(66.3) 
ood 
/ -39.1- 
(857 } 
38.0 
8 -27 .1- 
(27540) 
26.0 
9 -20.3- 
Z0b3)) 
20.0 
10 


Weapon Type 1 


550 


ike © 


- - denotes MINOS variables 


( ) denotes XS variables 


-51.1- 
(5 lea) 
51.0 


No brackets denotes XS integer variables for Problem 12. 
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Total 


-100 .0- 
(100.0) 
100 .0 








Target 


j 1 2 6 4 5 
ie) -33.2- 
fe3.2 <2 ) (eS) 
22.0 18.0 
12 -40 .9- 
(40.5) 
40.0 
13 -54 .0- 
(52.4) 
54 .0 
14 -58.8- 
(9.6) (Acres) 
10.0 42.0 
15 -26.2- -43 .8- 
(26.5) (43.5) 
26.0 44 .0 
16 -24.2- -17.0- 
(lara) (34.0) 
7.0 46 .0 
ie -3.8- -/2.0- 
(051) (52.1) 
18.0 27) 
18 -57 .6- 
(14.4) (46.0) 
58 .0 
19 -64 .2- 
(63.3) 
64 .0 
20 -62.4- 
(60.4) 
61.0 
Totals -200- -100- -300- -150- -250- 
(200) (100) (300) (150) (250) 
200 100 300 150 250 


- - denotes MINOS variables 
( ) denotes XS variables 
No brackets denotes XS integer variables for Problem 12. 
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Problem 7 (Waterman 7) 


Source: [Ref. 27] 
No. of variables: 100 
No. of constraints: 1 linear constraint 
|] nonlinear constraint 
50 lower bounds on the variables 


Objective function: 


50 
B. (x. 0) 
minimize: f(x) = >. oo 
1=1 


Bi(x, + 50) = 5 (si * YO %,| ; _ "%, 


and 
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~~ 
t-—— 
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K, = 300 
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Data from Problem 7 
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Results for Problem 7 
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Initial MINOS XS 


bi 300.0 432.01 602.37 
Xoo 300 .0 500.00 462.028 
X30 300 .0 192.82 187.38 
Xoy 300.0 153.31 139.08 
a 300.0 169.95 153.28 
XB 300.0 221.19 211.62 
Xan 300.0 155.56 146 .33 
X36 300 .0 290.95 258 .89 
X36 300 .0 272.94 242.92 
Xa 300 .0 300.04 464.12 
X30 300.0 230.70 220.40 
Xaq 300.0 283 .96 464.12 
Si 300 .0 279.00 464.12 
7) 300 .0 248.16 22703 
oe 300 .0 208 . 39 186.85 
re 300.0 249 .02 233.13 
A 300.0 241.68 228.57 
ais 300 .0 130.00 118.90 

: ae 300.0 239.61 242.71 

: Xay 300 .0 249 .04 464.12 
ie 300.0 409 .47 602.37 
i 300.0 146 .09 132.06 
a 300 .0 158.39 144 .48 
nF 300 .0 297 .80 304.94 
eg 300.0 334 .36 337 .09 
ae 300 .0 554.47 556.60 
- 300.0 S0gn 4) 323.76 
See 300.0 2227 208 .06 
Sag 300.0 543.68 | 544 .60 
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Xo 300.0 
X93 300.0 
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Problem 8 (Waterman 8) 


Soumce: ([Ref. 28] 
No. Of variables: 46 
No. of constraints: 1 nonlinear inequality constraint 
1 linear equality constraint 
46 lower bounds on the variables 


Objective function: 





46 
x Xs 
minimize: f(x) = = In 
1=1 
Constraints: 
46 
xX. 
2 > 
1=1 
46 
y | Cry as ad sy. < § 
i=l 
where ee + Xj 
j ¢ A(i) 


A(i) consists of all arcs (in Figure 7) that converge directly 
and indirectly upon node i. 


x, 20 teow et ao 
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Data from Problem 
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Fig. 7 Transportation Routes for Problem 8 
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Results for Problem 8 
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Problem 9 (Dean 9--Game) 


Source: Author 
No. of independent variables: 


No. of constraints: 


Objective function: 





maximize: X17 ° Xo 
Constraints: 
xe 
l + 
2 
(30) 
Xy2 X%_ 2.0 


Results for Problem 9 


Initial 
Gx.) 0.0 
x1 0.0 
Xo 40.0 
91 (x) 3.0246 





2 


1 nonlinear equality constraint 


cz De. 
(23) 
MINOS X$ 
345 .00 344 .94 
CUAL er0ok 
16.263 lOe 290 
1.0000 0.99889 
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Problem 10 (Dean--Sortie) 


Source: [Ref. 37] 
No. of independent variables: /93 
No. of constraints: 81 linear inequality constraints 


Objective function: 


J 
Keene 
» 5K; D5) 
oa 
where: I 
i C 
J J 
K. =—~ { 1 - exp | - — os rege ee 
J C; iF J me i) 1d 
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J 
> i Sy) re, »/ 
jt 





110 





lv 


1A 


all 1 and j 
0,943 (hoy Baa 
O94 2 ese? .. 5s 
1 J, 
i 3] 
7 2 
7 ioe -lie 13 
16,19,20,2/7 
bi 2324.25.20 
i eanco. 25.26 
7} J 
2 BO 247 351 
53,00 ,61 
8 30547 51 
53,007,601 


ial 


woe Ofesice constraints 


, no. of side constraints 


0.02 
O02 
WEOZ 


OEUZ 
RUZ 





Problem Data 


-—O 
anal « 
qu. 
onal 
ul 
~ 
oned 
u 
NO 
ond 
ul 
Cw 
— 
ul 
> 


Ce Ce Ce Cre Cre Car. Care Cs. Cos. Cre Care Ce Css Css Cs. Coe Cs. Cs. Cs. Cs. Cs, Cos. Cs, Cos. Cs. Cs. Cs. Cs. Cs. Cs. Cus. Cus. Cus. Cs. Cus. Cus. Cs, Csr. Cs. Cs. Cs. Cs. Cs, Cs. Cu. 


APH HHH 
nN & WPF © 


RRR RRR RRR Re 
DWONHANAWMHYFYOWONDOAW PH 


PO 
j— 


WW PO PO PO PO PO POS PY PO 
OW O™NDM MN & W PO 


GW G OO 
Wh Fe 


Co OG 
On & 


GW G Ww W 
OOoOmnN”A 


DOOdDDDODADADADIODDODIANAOADAOAOOODAOOOOO9OD0O0000 


DODODDDADMDADIMDOKRFRODOADODOAODODADAOODOODOADAOODOOOODOOOCOOO OO O009090 0000 
OO 
© 
Oo 
PO 


t— PO 
tr Om 
OO WG 
Mm © 


.cooV 
3810 
. 3409 
.3416 


© © 


OOO00 © 


2907 
. 2806 


See oqooqooqqoqgoqoqoqoqoqoqQooqoooooooooqooooOooooooeooaooooooooo 


ODDDDADDIDOADOOFRODADAODADIAOADADADADAOVDAOAODODAMOAOOAOOAOOOOOOOO900O00 0 


477 
.0344 
.2877 
. 2948 
.4693 
.0616 
eo] 
.0406 
sOoi/ 
.6140 
70520 
g8559 
.0403 
25/82 
a SS/ 
lize 
On 9 
s0eav 
.6347 
.1170 
.0024 
1/950 
.0058 
sOnZ) 
.0121 
.0083 
Zui 
20920 
26053 
. 1662 
.1401 
25005 
729 
S50 
O93 
meg7 2 


joes 
. 3888 


1888 
.3491 


ees Oem Ge OoQo qq qQoqoqgaegooaOooamooaooaooaooaoomooaoooeoOaooaooaoooooooodooOo © 


.6705 
.0676 
.4399 
.4482 
. 5089 
.0616 
nies 
.0639 
.0931 
./569 
.0827 
sS059 
.0658 
POO hS 
.2514 
pe07 2 
. 1930 
.0997 
.6347 
.2149 
.0039 
. 5080 
.0100 
.0221 
.0221 
eos 2 
Bou c 
mlz 
50053 
7ogU/ 
.1401 
USS 
JJo06 
.2810 
.9057 
OT 


solo 
. 3888 


wo2G 
. 349] 





“UO 
=. 
ci. 


Cos, Coss Cos Cos, Cos, Cote Cos, Coss Coss Cs, Cs, Cas, Cas. Cs, Cos, Cs. Cos, Cs. Cos, Cs, Cs. Cs. Cs. Cs. Cs, Cus. Cs, Cs. Cu, Cu, Cs. Cs, Cu, Co, Cai Cea Gai Ca a Caan Cu, 


nnn & @ ® Wm Wm Wm a aaa at aa 


WOON MDMN WM Fr 


OOQOoooao ooo oaooocoooooaooooooOorocoOrOCOOQGOoOOCOOOOOOoOOoOOoOOUOOUOOoOO OO 


. 1286 
. 1104 
.0361 
.0318 
.0080 
.0616 
age 
.0406 
osu 
.4104 
UZ 
089 
IZ30 
BU SOV) 
.0030 
.0510 
sod 
.0157 
Ze? 
. 3248 
.0024 
.4122 
.0044 
0032 
.0052 
.0034 
. 1404 
.0877 
.8078 
2472 
.1401 
,3033 
wcec 
13:2) 
.8938 
eoarc 


ges 
. 3888 


la / 
3491 


DOODOODOOOOO OOO OOO ONOOOO OOOO OCOOOOOOOCOOCO 


Swe eo oqgooooOaoooaooegoocoeooooooooeooooooooooooooooOoOOoO Oo 


Ms 


Oooo0oo0ordddododooooooo0o0o0o00co0 0rFRCVAOOOONOOOOO OO 0O00O00000 0 


. 1861 
. 1050 
.0564 
5050/ 
.0160 
.0784 
.0610 
.0507 
.0599 
.4493 
POMS 2 
sae? 
s030c 
e091 
.0030 
Uo9 
. 1904 
n0Z0y 
.8565 
.3184 
.0024 
.3749 
.0036 
.0052 
.0052 
.0034 
o£ 909 
. 1056 
.8078 
.2464 
.1413 
200) 
. 3044 
. 3000 
7 SOL 
.4931 


.0000 
S02 r/ 


.2656 
~4574 





“ww 
=). 
Co. 


Cy Cure Cr. Cr. Cs. Cs. Cor. Cr. Cor. Cur. Cur, Cur. Cur. Cur, Cur. Cur, Cur. Cu, Cur. Cur. Cur. Cur. Cur. Cur. Cur. Cur. Cur, Cur, Cus. Cus, Cus. Cus. Cus. Cus. Cus. Cur. Cus, Cur. Cus. Cus. Cus. Cs. Cs. Cs. Cs. 


hah a a a at 


Oo ONAN BP WPM re 


Gia @ QQqgqoaoeooqooqooqoooqoooqgmqQqooqooqoooaoqQooOoOOoOooOoOOoMOOoO Oo emM OOOO OoOOoORUKUlCl OU OUlCOCW OC LO 


ey 
.0244 
0723 
. 0686 
0212 
.0617 
.0716 
.0486 
0590 
4330 
. OMS2 
.4285 
.0285 
vS3 
.0040 
.0574 
.0648 
.0203 
6087 
. 1076 
.0024 
oon 
.0036 
OBZ 
.0052 
.0028 
. Lonel 
. 1061 
.0047 
.1314 
.0660 
aoa 
0610 
.0704 
.6724 
. 2858 


OveS 
. 4618 


. 1248 
nooo) 


ODDODADADADODAOUADAOAAOAOOAOIAOODODAOOO OCF DAOADODAOAGAAOOOOOOONO0O 00000000 


7aac3 
.0644 
.4157 
.4240 
-4546 
.0617 
35 
.0622 
.0888 
.7112 
.0903 
.4285 
.0647 
.5443 
ao 1.1 
285 
cule / 
0883 
.6087 
. 2049 
.0037 
.4168 
.0088 
.0205 
.0205 
sOs9 
. 3435 
ekS 1 
.0047 
woeOU 
.0660 
saul 
29290 
. 2658 
.8096 
.ceo9 


fei 23 
.4618 


WSS 
» 2985 


114 


OODADADADADADAMDAOAADAADADADADAAODADADOOFODOADAODADAOOCODOOCOOOOOOOOOQO0O00 0 


. 1818 
. 1036 
aoc] 
.0690 
mZ2Z 
.0629 
507 33 
res 
.0613 
.4467 
OZ 
-4385 
.0296 
. 1160 
.0100 
.0585 
.1795 
CAUS, 
a2022 
oLO/ 
.0024 
.2910 
.0036 
aWO52 
s0052 
.0028 
oy 0 
. 1099 
. 8096 
coal 
.0660 
e20GU 
3067 
goo 
"Og 
#2993 


mw03Z 
.4761 


© 


n2a22 
.2954 





C_, Cs. Cs. Cor. Cs. Csr, Cs. Cr. Cs. Cor. Cr, Cr. Cu. Cs. Cs, Co, Cu. Cu. Cu. Cu, Cur. Cu. Cu. Cu. Cu. Cu. Cu. Cu. Cu. Cu. Cs. Ca, Cu. Cu, Cu, Cu. Cu. Cu, Cu, Cu, Cu. Cu. Cu. Cs. Cu, 


=i: 
rs 


nunnet & W Ht WW th tt tn tb & tm nt tf tm ot th tm nb nb th th th om nh on mh on mh hh th th th ht ah th 


NR RE RRR eee ee 
ODwWOmn7wa nF WNYFM OW OANA ON PWN F 


NO RO RO PRO AO APO PO 
SHOP WN 


©) AO AO 
CO WO © 


W WW & WW OW 
SOHO PWM 


Pp PP PW W 
hom © WO ©} 


ppp 
On & Ww 


eS © @ ae © oa oo oa O@Go Oooo OOOrFrOOOd0d OO oOo oo oo ooo oO OO Om 


1h 


2009 
~1145 
soc) 
. 3900 
~4765 
Bilarae 
0 
.0942 
Or 
./068 
seo 
.4818 
/G502 
.5091 
.2284 
. 1434 
. 1/706 
0G 
s7ob2 
. 3022 
.0030 
. 3606 
.0079 
S017. 2 
S04 2 
ON 
0 
- 20 
.8081 
2005 
AS oo 
oe0 
.2816 
.1974 
.0607 
food 


.0000 
#6659 


2550 
-4726 


SoC @ @ee@ @eqQ OOqQOoOe e@omaomaoooaooooooooOodOoooOooOOoOoOOoOOoOOoOOoOoOoOoO oO 


.0360 
.0070 
.0070 
.0060 
.0110 
.0760 
.0260 
. 0650 
. 0430 
. 1800 
.0080 
iS 7.0 
.0350 
.0390 


.0390 
.1210 
.0110 
~4230 
eo 0 
.0020 
0920 
.0040 
.0050 
.0050 
. 0040 
.0870 
.0610 
.0020 
.0498 
mer50 
. 1540 
.0500 
.0490 
.0220 
2500 


OO © 


15 


ODOODDADDDDADADDADADOAODDAOADVDADDAOAOODODOOODOOOAOODODOOOOO0O0O000 © 


MH OOdDDODOO0O00 WO 


OOOODDODAOVDODAODOOOOOOOOO O00 000000 © 


Cw 
f= 
©O 
eS) 


. 1053 
a2 
. 1/82 


483 


782 





oO 
alls 
Cus. 


Ca Cn Ca Ca Cs Ca Cai a Ca Ca Ca Cas C Ges. Cs. 


~~ 
aad. 
C4. 


Cag Ca Ce Cs Cae Ces, Cons Cr Cay Cars Cass Car, Cos, Cos. Cos, Cos. 


new Wot &@ nt th ot ot ott ot ott ot oth oth 


neu Ww ht tt tt & Ht ow ot tH Mh ow tt 


46 
47 
48 
49 
90 


a2 
o3 
54 
SO 
96 
o/ 


oo 
60 
61 


46 
47 
48 
49 
90 
ol 


2 
54 
Sho 
96 


58 
99 
60 
61 


O313/ 
0.0876 
0.0819 
0.0 

O7es9 
ONZ500 
0.8162 
0.2569 
1.3428 
1.3680 
0.0 

0.4781 
0.0805 
0.8097 
OZone 
0.0772 


052/07 
0.2337 
0.0819 
0.5366 
0.8148 
0.4076 
0.8176 
0.6316 
1.2897 
1.2234 
O50 

075573 
0.0798 
0.0895 
0.2466 
Orin) 


ODDADAGUAUADADAOOO9OD0O OOO 


ODDDDAOVUDAOAAOADAAOAADAOAAOOAO © 


. Loe] 
foo) 
noo3 


. 1934 
.4606 
.4058 
7 O0u 7) 
Bwloou 
soe 


nOOy 3 
.0277 
. 0637 
wees 
lez 


OODODDAONDDADVDODAOOAOAO OO 


116 


ODDDADADAAUDAOAAADAOAOOOAOO 


OVDDdDODODADADADAAMDAOAOAO0O0O 


. 1856 
seg! 
.0471 
. 1641 
. 1934 
.1504 
.4058 
. 1804 
./030 
.9612 


S557.5 
.0457 
.0637 
.1421 
.1120 


. 2683 
2421 
.0823 
#21/.6 
fol 
-4276 
eo) 
i000) 
qi 0 | 
./924 


.6876 
.0/799 
.8100 
.2469 
U162 


So Ooo OOOO OO © © © 


sG065 
neo ST, 
ORY 
. 1641 
.1934 
a2040 
.4058 
7029 
730 
e207 


soc 
Og 
.0637 
2000 
.1120 





0 
=. 
Cua. 


Cua. Cua. C5, Coase Cua. Coa. Cos, Cos, Cus, C4, as Cua, Cua, Cua, Cus, Cus, 


“OO 
amd» 
Cs. 


C5. Cos, Goa. Gua Coa. Cos, Coa Cre. C., Gu. Coa Ca Ca Ca Ca. 


buuw ub wb UW th Ww WwW UW Ww th wh wb woot 


bow uk haa a 


46 
47 
48 
49 
50 
a1 
oc 
S3 
54 
39 
56 
a7 
58 
5g 
60 
61 


46 
47 
48 
49 
90 
ol 
Bc 
53 
54 
oo 
96 
ay 
98 
oo 
60 
61 


OdDod dod dc OO 000000 © 


OOddd0dcdodo0o0o 000000 


.1324 
aeZbS 
.0284 
. 1443 
. 1803 
. 3479 
. 3806 
5443 
.6039 
.9474 


FOLaS 
OHS 2 
7Ud00 
. 1643 
.0608 


1 = ie 


.2465 
. 2641 
.0842 
~4547 
.8714 
5510 
. 8387 
-/05/ 
.Oguc 
.6240 


7425 
.0827 
.8134 
.2914 
~1245 


Oooo c0odtrrODOODO0O00 © 


ODooo0od0d0d0 20000000 


. 3446 
E203 
.0920 
1443 
. 1803 
NCIC 
. 3806 
. 3496 
~4349 
. 3944 


.5145 
.O911 
.0565 
. 2804 
.0697 


. 1620 
.0640 
.0344 
.1790 
2243 
~ 1415 
518 
.28/0 
. 1900 
. 1460 


we2c0 
20176 
.0245 
0772 
.0348 


ie; 


Odooc0cdrDAOdDOdDOdDO0O00 O 


OOodo 0nd dado d0O0000 QO 


. 2868 
ei s22 
.0844 
.4848 
.8204 
«20/6 
.8244 
-4842 
noe) 
aS 


-oeo/ 
.0815 
mote? 
G00 
.0646 


.1782 
.0745 
.0745 


. /630 
x2o08 
wou 
12300 


. 3896 
.0340 
. 3483 
«1053 
.0340 





Problem Data 


Cu. Co. Cu. Cu. Cur. Cus. Cur. Cus, Cas. Cos. Care Cas, Cause Cus, Cs. Cs. Cs. Cas, Cs, Cs. Cs, Cs, Cs. Cs. Cas, Cs. Cs, Cs, Cas. Cs. Cas, Cs. Cs. Cs. Cs. Cas, Cs. Cs. Cs. Cs. Cs, Cs, Cas, Cs. Cs. 


hoe ht a at 


iO CO™ AN & W Mr 


900 
500 
900 


500 
900 
900 


300 
300 
300 
300 
300 
300 
300 
300 
900 


500 
900 
500 
900 
900 
900 
900 
900 


900 
900 
900 
1500 
1500 


900. 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 


500 
900 
500 
500 
700 
700 
700 
700 
700 
700 
400 


.0000 
.0000 
.0000 
900. 
.0000 
. 0000 
.0000 
900. 
900. 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
900. 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
900. 
.0000 
.0000 
.0000 
.0000 
.0000 


0000 


0000 
0000 


0000 


0000 


0000 


aoa ao OO QOOOOooaoOooaocoO OOOO OoOOCOOOOOO OO OOO OOO OOM OOO OOO 
So Sooo SOQ QomIoaoaoaomtaomiooaoaoonooaooaooaooamaooooonmuoaomtoo°onooooOoOoOllOoOO Om 


f— 
} 
Oo 


ODODOOODVDADADUDUAOUDAIUDVDWDAODINDAUDUADODAOUDIWDVOUIONAOANOAOADVDAOODOODVAONOOOOOOO0O00O 
DOOODAOABDADVDAOUOAODAUDUUDAUBDABDIDUDIDOUDAUODODODUDIUAUAIAONOOAODDAMOAOOOONOOOOOO0000 QO 


ODODOADDODAUDUUDAUDADAUDUDAUAUDUDAOBDAUDAUAOUVDAUDUWUIUAODMDAUOAUVDOUAOOUAONOAOOOOODOAOOOO OS 


. 1000 
.2500 
mou 
. 1500 
. 2000 
. 2000 
. 1000 
. 1500 
. 1000 
. 1000 
. 1000 
. 1000 
. 1000 
.5000 
.5000 
. 1000 
. 2000 
. 2000 
. 5000 
.4500 
. 5000 
. 1500 
.5500 
.9000 
. 5000 
. 5000 
. 1000 
. 1000 
. 3000 
.2500 
.5000 
. 3000 
.4000 
-4000 
-4000 
4000 
. 4000 
.4000 
.4000 
. 2500 
.2500 
.4000 
.4000 
.2000 
.2000 


— bb tt 


me Pe 
mr © ©O © 


— — — ee re — rt Mm em POF FF RO MR rR re tr 
Oo nna NN NNR ORK rR rR OONMODWODODOMNUWDMDOOO0O OF MN 


. 0000 
.0000 
.0000 
.0000 
. 0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
. 0000 
.0000 
.0000 
.0000 
.0000 
.0000 
. 0000 
. 0000 
. 0000 
. 0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
. 0000 
.0000 
.0000 
.0000 





Ca aa, Car, Car, rear ear. a1, ee. Coon, Cea Ca Gas Coa, Cu, 


400 
400 
400 


400. 
.0000 
.0000 
.0000 
.Q000 
900. 
.0000 
.0000 
.0000 


400 
400 
5000 
500 


900 
500 
900 


900. 
900. 
.0000 
.0000 


300 
300 


.Q000 
. 0000 
. 0000 


0000 


0000 


0000 
0000 


Oooo 0 cd0cdc O02 O00 0000 
Sree @ Oo 2 OQ Oo QO @ @ @ © 


119 


Xe 2 2 @ 2 2.2 oC Oo @ @ a a2 © 
ooo Ooo © OO 200000 0 © 


ODonoddc00 00 O00 0000 


. 2000 
. 2000 
. 2000 
-4000 
ou 
e200 
. 3500 
. 3000 
-4500 
.4500 
~4500 
ecoU0 
.4000 
. 2000 
. 2000 
.4000 


10 


10 
10 
20 
10 


10 
10 
10 
10 
10 


.0000 
. 0000 
.0000 
.Q000 
.0000 
.0000 
.0000 
. 0000 
.0000 
.0000 
.0000 
.0000 
.0000 
.0000 
. 0000 
.0000 





Results for Problem 10: (XS) 


(only non-zero values of final solution shown) 
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Problem ll 
See Section II1.C.1l. 


Problem 12 


See Problem 6. 
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Problem 13 (Dean--Integer Nonlinear) 


Source: [Ref. 38] 
No. of independent variables: 12 (8 binary) 
No. of constraints: 48 nonlinear equality constraint 


3 linear equality constraint 


Objective function: 
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minimize: o.(26.n. ) 


1=1 


Constraints: 
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Results for Problem 13: 
(only final solution shown) 
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